Class cascading.RegressionPipesPlatformTest

18

tests

0

failures

0

ignored

19.111s

duration

100%

successful

Tests

Test Duration Result
testCoGroupSplitPipe 2.344s passed
testComplexLogicAnd 0.499s passed
testComplexLogicOr 3.822s passed
testComplexLogicXor 0.477s passed
testCopy 0.480s passed
testDeepPipeline 4.402s passed
testDupeHeadNames 0.055s passed
testDupeTailNames 0.064s passed
testGroupBySplitPipe 1.405s passed
testGroupNoneSortUnknown 1.128s passed
testIllegalCharsInTempFiles 0.715s passed
testLastEachNotModified 0.610s passed
testOOMEPostGroup 0.772s passed
testOOMEPreGroup 0.474s passed
testUnGroupUnknown 0.407s passed
testUnknown 0.444s passed
testUnknownReplace 0.535s passed
testVarWidth 0.478s passed

Standard output

2016-05-18 01:32:54,383 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 01:32:54,417 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.RegressionPipesPlatformTest, with platform: hadoop2-tez
2016-05-18 01:32:54,454 INFO  tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 01:32:55,056 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 01:32:55,386 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides: 
2016-05-18 01:32:55,663 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:55,665 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:55,757 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: 8440BE39258F4F0AB26054026672DE42
2016-05-18 01:32:56,008 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.102
2016-05-18 01:32:56,021 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.116
2016-05-18 01:32:56,022 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:56,023 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:56,024 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:56,208 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:32:56,214 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor
2016-05-18 01:32:56,431 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 01:32:56,434 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:32:56,435 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:56,435 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"]
2016-05-18 01:32:56,436 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:32:56,436 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:32:56,436 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:32:56,437 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...regression/complexlogicor
2016-05-18 01:32:56,438 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:56,460 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:32:56,634 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:56,640 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:56,641 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:56,706 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001 doesn't exist and is created
2016-05-18 01:32:56,800 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:56,801 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:56,802 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd
2016-05-18 01:32:56,868 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535176640_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:56,901 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:56,901 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 01:32:56,901 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 01:32:56,902 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:56,902 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:56,915 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:56,941 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:56,969 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:56,970 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:56,971 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:56,972 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:56,973 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:56,973 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:56,974 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:56,975 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:56,976 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:56,976 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:56,987 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:56,990 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:57,002 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 01:32:57,002 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 01:32:57,002 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:57,015 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:57,018 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:57,018 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:57,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir
2016-05-18 01:32:57,022 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/history.txt.appattempt_1463535176640_0001_000000, maxErrors=10
2016-05-18 01:32:57,022 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:57,024 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535176640_0001_000000, appSubmitTime=1463535176846, launchTime=1463535176866
2016-05-18 01:32:57,031 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:57,092 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40193
2016-05-18 01:32:57,094 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535176640
2016-05-18 01:32:57,103 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 01:32:57,103 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 01:32:57,103 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:57,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535176640_0001_000000, startTime=1463535177113
2016-05-18 01:32:57,114 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:57,203 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:57,220 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:57,220 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001, dagName=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,432 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,440 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:57,494 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535176640_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/dag_1463535176640_0001_1.dot
2016-05-18 01:32:57,498 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/dag_1463535176640_0001_1-tez-dag.pb.txt
2016-05-18 01:32:57,534 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535176640_0001_1
2016-05-18 01:32:57,534 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:57,534 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57 Running Dag: dag_1463535176640_0001_1
2016-05-18 01:32:57,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535176640_0001_1, submitTime=1463535177432
2016-05-18 01:32:57,614 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001/tez-conf.pb" } size: 40938 timestamp: 1463535176000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:57,614 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,615 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,615 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:57,617 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:57,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535176640_0001_1, initTime=1463535177619
2016-05-18 01:32:57,620 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:57,624 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:57,627 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_STARTED]: dagID=dag_1463535176640_0001_1, startTime=1463535177627
2016-05-18 01:32:57,628 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:57,628 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E8FEF806C7D54B23A902AB6C1336327C : {6F1E22DC3F75464282FD5A0697F8F54A={InputName=6F1E22DC3F75464282FD5A0697F8F54A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:57,630 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 6F1E22DC3F75464282FD5A0697F8F54A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:57,630 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,639 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,640 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,641 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001, dagName=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,643 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,652 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 6F1E22DC3F75464282FD5A0697F8F54A on vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,681 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535176640_0001, with dag id: dag_1463535176640_0001_1
2016-05-18 01:32:57,686 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:57,686 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 6F1E22DC3F75464282FD5A0697F8F54A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:57,719 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:57,719 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:32:57,720 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:57,721 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:57,737 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 6F1E22DC3F75464282FD5A0697F8F54A on vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,745 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:57,746 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {6F1E22DC3F75464282FD5A0697F8F54A=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:57,759 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] parallelism set to 1
2016-05-18 01:32:57,759 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:57,794 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], numAdditionalOutputs=1
2016-05-18 01:32:57,794 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=87340D55D24749218144C739866F0086, vertex=vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:57,803 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E8FEF806C7D54B23A902AB6C1336327C:87340D55D24749218144C739866F0086 using oldmapred API
2016-05-18 01:32:57,804 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=87340D55D24749218144C739866F0086, vertexName=E8FEF806C7D54B23A902AB6C1336327C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:57,815 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, vertexId=vertex_1463535176640_0001_1_00, initRequestedTime=1463535177628, initedTime=1463535177794, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:57,815 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] with distanceFromRoot: 0
2016-05-18 01:32:57,815 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:57,817 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535176640_0001_1_00, startRequestedTime=1463535177686, startedTime=1463535177815
2016-05-18 01:32:57,818 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535176640_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:57,819 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:57,820 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E8FEF806C7D54B23A902AB6C1336327C
2016-05-18 01:32:57,820 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,826 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_STARTED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskId=task_1463535176640_0001_1_00_000000, scheduledTime=1463535177824, launchTime=1463535177824
2016-05-18 01:32:57,849 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,891 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:57,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535176640_0001_00_000001, launchTime=1463535177907
2016-05-18 01:32:57,912 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535176640_0001_00_000001
2016-05-18 01:32:57,916 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535176640_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57 Starting to run new task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,917 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,918 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:57,919 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:57,918 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535176640_0001_1_00_000000_0] started. Is using containerId: [container_1463535176640_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:57,920 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, startTime=1463535177916, containerId=container_1463535176640_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:57,921 INFO  counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters:  GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 01:32:57,933 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, VertexName: E8FEF806C7D54B23A902AB6C1336327C, VertexParallelism: 1, TaskAttemptID:attempt_1463535176640_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6F1E22DC3F75464282FD5A0697F8F54A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=87340D55D24749218144C739866F0086, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:32:57,935 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:57,939 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,948 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535176640_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:57,950 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535176640_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:57,973 INFO  input.MRInput (MRInput.java:initialize(440)) - 6F1E22DC3F75464282FD5A0697F8F54A using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:57,974 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:57,975 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:57,991 INFO  output.MROutput (MROutput.java:initialize(421)) - 87340D55D24749218144C739866F0086: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:32:57,996 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, ordinal: 0
2016-05-18 01:32:57,997 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, mem on start (mb), free: 243, total: 281, max: 672
2016-05-18 01:32:57,997 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:57,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:57,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:57,998 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:57,998 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6F1E22DC3F75464282FD5A0697F8F54A:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [87340D55D24749218144C739866F0086:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:32:58,003 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:58,007 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:58,007 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:32:58,007 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6F1E22DC3F75464282FD5A0697F8F54A:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [87340D55D24749218144C739866F0086:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:32:58,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:58,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:58,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 6F1E22DC3F75464282FD5A0697F8F54A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:58,009 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:58,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:58,010 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:58,012 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,028 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:32:58,029 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 6F1E22DC3F75464282FD5A0697F8F54A initialized RecordReader from event
2016-05-18 01:32:58,047 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:58,062 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:58,063 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"], id: 87340D55D24749218144C739866F0086
2016-05-18 01:32:58,063 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"]
2016-05-18 01:32:58,067 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:58,069 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:32:58,078 INFO  output.MROutput (MROutput.java:close(526)) - 87340D55D24749218144C739866F0086 closed
2016-05-18 01:32:58,082 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535176640_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:32:58,084 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,084 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, mem on close (mb), free: 237, total: 281, max: 672
2016-05-18 01:32:58,085 INFO  output.MROutput (MROutput.java:close(526)) - 87340D55D24749218144C739866F0086 closed
2016-05-18 01:32:58,086 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,087 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:32:58,088 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535176640_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:32:58,091 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535176640_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4][cascading.flow.SliceCounters Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=4]]
2016-05-18 01:32:58,096 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535176640_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:32:58,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, creationTime=1463535177842, allocationTime=1463535177901, startTime=1463535177916, finishTime=1463535178095, timeTaken=179, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:32:58,102 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,104 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,105 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:32:58,105 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,107 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskId=task_1463535176640_0001_1_00_000000, startTime=1463535177916, finishTime=1463535178103, timeTaken=187, status=SUCCEEDED, successfulAttemptID=attempt_1463535176640_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4
2016-05-18 01:32:58,116 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:32:58,116 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:58,133 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, vertexId=vertex_1463535176640_0001_1_00, initRequestedTime=1463535177628, initedTime=1463535177794, startRequestedTime=1463535177686, startedTime=1463535177815, finishTime=1463535178117, timeTaken=302, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4, vertexStats=firstTaskStartTime=1463535177916, firstTasksToStart=[ task_1463535176640_0001_1_00_000000 ], lastTaskFinishTime=1463535178095, lastTasksToFinish=[ task_1463535176640_0001_1_00_000000 ], minTaskDuration=179, maxTaskDuration=179, avgTaskDuration=179.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535176640_0001_1_00_000000 ], longestDurationTasks=[ task_1463535176640_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:32:58,134 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:32:58,137 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535176640_0001_00_000001, stoppedTime=1463535178136, exitStatus=0
2016-05-18 01:32:58,137 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:32:58,138 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,138 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535176640_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:32:58,138 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535176640_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:32:58,139 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535176640_0001_00_000001, Exiting
2016-05-18 01:32:58,139 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,139 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,140 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535176640_0001_1
2016-05-18 01:32:58,141 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535176640_0001_1
2016-05-18 01:32:58,142 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535176640_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:32:58,143 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535176640_0001_00_000001 completed successfully
2016-05-18 01:32:58,143 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:32:58,143 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 87340D55D24749218144C739866F0086 for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], outputName: 87340D55D24749218144C739866F0086
2016-05-18 01:32:58,144 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463535176640_0001_00_000001
2016-05-18 01:32:58,161 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:87340D55D24749218144C739866F0086 of vertex/vertexGroup:E8FEF806C7D54B23A902AB6C1336327C isVertexGroupOutput:false
2016-05-18 01:32:58,161 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,169 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535176640_0001_1, queueSize=0
2016-05-18 01:32:58,172 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535176640_0001_1, startTime=1463535177624, finishTime=1463535178161, timeTaken=537, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1280, AM_GC_TIME_MILLIS=24, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4
2016-05-18 01:32:58,174 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535176640_0001_1 finished with state: SUCCEEDED
2016-05-18 01:32:58,174 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:32:58 Completed Dag: dag_1463535176640_0001_1
2016-05-18 01:32:58,176 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535176640_0001_1, dagState=SUCCEEDED
2016-05-18 01:32:58,179 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535176640_0001_1_post
2016-05-18 01:32:58,179 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,181 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:32:58,181 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:32:58,186 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, with id=dag_1463535176640_0001_1
2016-05-18 01:32:58,188 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001
2016-05-18 01:32:58,191 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, with id=dag_1463535176640_0001_1
2016-05-18 01:32:58,197 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:32:58,197 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:32:58,197 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:32:58,197 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:32:58,198 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:32:58,201 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:32:58,203 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:32:58,203 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:32:58,204 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:32:58,204 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,206 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:32:58,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:32:58,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,208 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:32:58,214 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001
2016-05-18 01:32:58,217 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor/_temporary
2016-05-18 01:32:58,217 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:01.781
2016-05-18 01:32:58,272 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor
2016-05-18 01:32:58,311 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:58,311 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:58,380 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 01:32:58,397 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.045
2016-05-18 01:32:58,398 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:58,398 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:58,399 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:58,423 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:32:58,424 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:58,471 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting
2016-05-18 01:32:58,472 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,472 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:58,475 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  parallel execution of steps is enabled: false
2016-05-18 01:32:58,475 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  executing total steps: 1
2016-05-18 01:32:58,476 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  allocating management threads: 1
2016-05-18 01:32:58,477 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting step: (1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,477 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [split] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:58,515 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tez session mode enabled: true
2016-05-18 01:32:58,520 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:58,521 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:58,521 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:58,531 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001 doesn't exist and is created
2016-05-18 01:32:58,568 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:58,568 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:58,570 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd
2016-05-18 01:32:58,589 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535178521_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:58,589 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:58,590 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:58,595 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:58,595 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:58,595 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:58,596 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:58,597 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:58,603 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:58,603 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:58,603 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:58,604 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:58,604 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:58,604 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:58,604 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:58,605 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:58,605 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:58,605 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:58,606 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:58,606 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:58,615 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:58,617 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:58,617 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:58,617 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir
2016-05-18 01:32:58,618 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/history.txt.appattempt_1463535178521_0001_000000, maxErrors=10
2016-05-18 01:32:58,618 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:58,619 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535178521_0001_000000, appSubmitTime=1463535178588, launchTime=1463535178588
2016-05-18 01:32:58,640 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:58,641 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54229
2016-05-18 01:32:58,644 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535178521
2016-05-18 01:32:58,652 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535178521_0001_000000, startTime=1463535178652
2016-05-18 01:32:58,652 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:58,669 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:58,669 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:58,669 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001, dagName=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,700 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,701 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:58,702 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535178521_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/dag_1463535178521_0001_1.dot
2016-05-18 01:32:58,704 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/dag_1463535178521_0001_1-tez-dag.pb.txt
2016-05-18 01:32:58,710 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535178521_0001_1
2016-05-18 01:32:58,710 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,711 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58 Running Dag: dag_1463535178521_0001_1
2016-05-18 01:32:58,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535178521_0001_1, submitTime=1463535178700
2016-05-18 01:32:58,731 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001/tez-conf.pb" } size: 40938 timestamp: 1463535178000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:58,732 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,732 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,732 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:58,733 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:58,735 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535178521_0001_1, initTime=1463535178733
2016-05-18 01:32:58,737 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:58,738 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:58,739 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_STARTED]: dagID=dag_1463535178521_0001_1, startTime=1463535178739
2016-05-18 01:32:58,739 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:58,739 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E0FD00F704B74F66B6AC601C3AA66EA4 : {E1A3E10480924DA68FAE85720A4CADB7={InputName=E1A3E10480924DA68FAE85720A4CADB7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:58,740 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E1A3E10480924DA68FAE85720A4CADB7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:58,740 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001, dagName=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,741 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,743 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:58,744 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E1A3E10480924DA68FAE85720A4CADB7 on vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,754 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E1A3E10480924DA68FAE85720A4CADB7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:58,759 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:58,759 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:32:58,760 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:58,760 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:32:58,762 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E1A3E10480924DA68FAE85720A4CADB7 on vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,764 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:58,765 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E1A3E10480924DA68FAE85720A4CADB7=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:58,766 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] parallelism set to 1
2016-05-18 01:32:58,767 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:58,768 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], numAdditionalOutputs=1
2016-05-18 01:32:58,768 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FFDAAF19BA3149E2BE99E4ACA1A46D82, vertex=vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:58,776 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E0FD00F704B74F66B6AC601C3AA66EA4:FFDAAF19BA3149E2BE99E4ACA1A46D82 using oldmapred API
2016-05-18 01:32:58,776 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FFDAAF19BA3149E2BE99E4ACA1A46D82, vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:58,780 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, vertexId=vertex_1463535178521_0001_1_00, initRequestedTime=1463535178739, initedTime=1463535178768, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:58,780 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] with distanceFromRoot: 0
2016-05-18 01:32:58,780 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:58,781 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535178521_0001_1_00, startRequestedTime=1463535178744, startedTime=1463535178781
2016-05-18 01:32:58,782 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535178521_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:58,782 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:58,783 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] submitted tez dag to app master: application_1463535178521_0001, with dag id: dag_1463535178521_0001_1
2016-05-18 01:32:58,783 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E0FD00F704B74F66B6AC601C3AA66EA4
2016-05-18 01:32:58,783 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,785 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_STARTED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskId=task_1463535178521_0001_1_00_000000, scheduledTime=1463535178784, launchTime=1463535178784
2016-05-18 01:32:58,785 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,789 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:58,795 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535178521_0001_00_000001, launchTime=1463535178794
2016-05-18 01:32:58,800 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,801 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535178521_0001_1_00_000000_0] started. Is using containerId: [container_1463535178521_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:58,802 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, startTime=1463535178801, containerId=container_1463535178521_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:58,803 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535178521_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58 Starting to run new task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,804 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,805 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,805 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:58,806 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, VertexName: E0FD00F704B74F66B6AC601C3AA66EA4, VertexParallelism: 1, TaskAttemptID:attempt_1463535178521_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1A3E10480924DA68FAE85720A4CADB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:32:58,806 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:58,808 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,812 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535178521_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:58,813 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535178521_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:58,819 INFO  input.MRInput (MRInput.java:initialize(440)) - E1A3E10480924DA68FAE85720A4CADB7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:58,847 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,850 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:58,864 INFO  output.MROutput (MROutput.java:initialize(421)) - FFDAAF19BA3149E2BE99E4ACA1A46D82: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:32:58,867 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, ordinal: 0
2016-05-18 01:32:58,867 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, mem on start (mb), free: 258, total: 281, max: 672
2016-05-18 01:32:58,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:58,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:58,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:58,867 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:58,867 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E1A3E10480924DA68FAE85720A4CADB7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FFDAAF19BA3149E2BE99E4ACA1A46D82:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:32:58,867 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:58,869 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:58,870 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:32:58,870 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E1A3E10480924DA68FAE85720A4CADB7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:32:58,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:58,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E1A3E10480924DA68FAE85720A4CADB7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:58,870 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:58,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:58,871 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:58,871 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,874 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:32:58,874 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - E1A3E10480924DA68FAE85720A4CADB7 initialized RecordReader from event
2016-05-18 01:32:58,878 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,881 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,882 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"], id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:58,882 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:58,883 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,883 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:32:58,888 INFO  output.MROutput (MROutput.java:close(526)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 closed
2016-05-18 01:32:58,889 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535178521_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:32:58,890 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,890 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, mem on close (mb), free: 255, total: 281, max: 672
2016-05-18 01:32:58,890 INFO  output.MROutput (MROutput.java:close(526)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 closed
2016-05-18 01:32:58,890 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,892 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:32:58,892 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535178521_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:32:58,893 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535178521_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:32:58,895 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535178521_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:32:58,895 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, creationTime=1463535178785, allocationTime=1463535178793, startTime=1463535178801, finishTime=1463535178895, timeTaken=94, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:32:58,895 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:32:58,896 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,896 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,896 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,898 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskId=task_1463535178521_0001_1_00_000000, startTime=1463535178801, finishTime=1463535178896, timeTaken=95, status=SUCCEEDED, successfulAttemptID=attempt_1463535178521_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:32:58,899 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:32:58,899 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,901 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, vertexId=vertex_1463535178521_0001_1_00, initRequestedTime=1463535178739, initedTime=1463535178768, startRequestedTime=1463535178744, startedTime=1463535178781, finishTime=1463535178899, timeTaken=118, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535178801, firstTasksToStart=[ task_1463535178521_0001_1_00_000000 ], lastTaskFinishTime=1463535178895, lastTasksToFinish=[ task_1463535178521_0001_1_00_000000 ], minTaskDuration=94, maxTaskDuration=94, avgTaskDuration=94.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535178521_0001_1_00_000000 ], longestDurationTasks=[ task_1463535178521_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:32:58,901 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:32:58,901 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535178521_0001_00_000001, stoppedTime=1463535178901, exitStatus=0
2016-05-18 01:32:58,901 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:32:58,901 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,902 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535178521_0001_1
2016-05-18 01:32:58,902 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535178521_0001_1
2016-05-18 01:32:58,902 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: FFDAAF19BA3149E2BE99E4ACA1A46D82 for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], outputName: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:58,902 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:32:58,904 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535178521_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:32:58,907 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535178521_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:32:58,907 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535178521_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:32:58,908 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535178521_0001_00_000001, Exiting
2016-05-18 01:32:58,908 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,908 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,910 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535178521_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:32:58,910 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535178521_0001_00_000001 completed successfully
2016-05-18 01:32:58,918 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:FFDAAF19BA3149E2BE99E4ACA1A46D82 of vertex/vertexGroup:E0FD00F704B74F66B6AC601C3AA66EA4 isVertexGroupOutput:false
2016-05-18 01:32:58,918 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,920 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535178521_0001_1, queueSize=0
2016-05-18 01:32:58,921 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535178521_0001_1, startTime=1463535178738, finishTime=1463535178918, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=470, AM_GC_TIME_MILLIS=27, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:32:58,921 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535178521_0001_1 finished with state: SUCCEEDED
2016-05-18 01:32:58,921 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:32:58 Completed Dag: dag_1463535178521_0001_1
2016-05-18 01:32:58,922 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535178521_0001_1, dagState=SUCCEEDED
2016-05-18 01:32:58,922 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535178521_0001_1_post
2016-05-18 01:32:58,922 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,922 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:32:58,922 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:32:58,922 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, with id=dag_1463535178521_0001_1
2016-05-18 01:32:58,923 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, with id=dag_1463535178521_0001_1
2016-05-18 01:32:58,929 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001
2016-05-18 01:32:58,933 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:32:58,933 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:32:58,933 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:32:58,933 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:32:58,933 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:32:58,934 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:32:58,936 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:32:58,936 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:32:58,937 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:32:58,937 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,937 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:32:58,938 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:32:58,938 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,939 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:32:58,943 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001
2016-05-18 01:32:58,945 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/_temporary
2016-05-18 01:32:58,946 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  completed in: 00:00.474
2016-05-18 01:32:58,989 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:58,990 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:59,132 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.083
2016-05-18 01:32:59,155 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.105
2016-05-18 01:32:59,156 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:32:59,156 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:32:59,157 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:59,184 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:59,209 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:59,233 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit
2016-05-18 01:32:59,383 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] starting
2016-05-18 01:32:59,383 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,383 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"]
2016-05-18 01:32:59,384 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  parallel execution of steps is enabled: false
2016-05-18 01:32:59,384 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  executing total steps: 1
2016-05-18 01:32:59,384 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  allocating management threads: 1
2016-05-18 01:32:59,385 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] starting step: (1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,385 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lhs*rhs] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:59,402 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tez session mode enabled: true
2016-05-18 01:32:59,406 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:59,407 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:59,407 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:59,417 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001 doesn't exist and is created
2016-05-18 01:32:59,442 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:59,442 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:59,443 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd
2016-05-18 01:32:59,459 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535179407_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:59,460 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:59,460 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:59,462 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:59,462 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:59,463 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:59,463 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:59,464 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:59,467 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:59,467 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:59,467 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:59,467 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:59,468 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:59,468 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:59,468 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:59,468 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:59,469 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:59,469 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:59,469 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:59,469 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:59,473 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:59,474 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:59,474 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:59,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir
2016-05-18 01:32:59,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/history.txt.appattempt_1463535179407_0001_000000, maxErrors=10
2016-05-18 01:32:59,476 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:59,476 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535179407_0001_000000, appSubmitTime=1463535179459, launchTime=1463535179459
2016-05-18 01:32:59,484 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:59,492 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33492
2016-05-18 01:32:59,492 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535179407
2016-05-18 01:32:59,496 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535179407_0001_000000, startTime=1463535179496
2016-05-18 01:32:59,496 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:59,542 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:59,543 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:59,543 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001, dagName=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,569 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,569 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:59,571 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535179407_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/dag_1463535179407_0001_1.dot
2016-05-18 01:32:59,573 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/dag_1463535179407_0001_1-tez-dag.pb.txt
2016-05-18 01:32:59,589 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535179407_0001_1
2016-05-18 01:32:59,589 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,589 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59 Running Dag: dag_1463535179407_0001_1
2016-05-18 01:32:59,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535179407_0001_1, submitTime=1463535179569
2016-05-18 01:32:59,607 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,607 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,608 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,609 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,609 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,609 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,610 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,611 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,611 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,620 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:59,620 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535179407_0001_1, initTime=1463535179620
2016-05-18 01:32:59,621 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:59,622 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:59,622 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001, dagName=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,622 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_STARTED]: dagID=dag_1463535179407_0001_1, startTime=1463535179622
2016-05-18 01:32:59,622 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BA80601CC7DA4457869BE0F2F8B5EC29 : {FFDAAF19BA3149E2BE99E4ACA1A46D82={InputName=FFDAAF19BA3149E2BE99E4ACA1A46D82}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FFDAAF19BA3149E2BE99E4ACA1A46D82, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,624 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:59,624 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B1A06BDE26DF401DB6A2AD73562926CF : {FFDAAF19BA3149E2BE99E4ACA1A46D82={InputName=FFDAAF19BA3149E2BE99E4ACA1A46D82}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:59,624 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FFDAAF19BA3149E2BE99E4ACA1A46D82, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:59,625 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,626 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:59,626 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,626 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,629 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,631 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FFDAAF19BA3149E2BE99E4ACA1A46D82 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:59,633 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FFDAAF19BA3149E2BE99E4ACA1A46D82 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:59,635 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:32:59,637 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,638 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,639 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:32:59,639 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], numAdditionalOutputs=1
2016-05-18 01:32:59,639 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=B5195B2CFFEB4747B400F2242B36B10F, vertex=vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:59,642 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AAD10383766F45E39B8107B0CA5AC336:B5195B2CFFEB4747B400F2242B36B10F using oldmapred API
2016-05-18 01:32:59,642 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=B5195B2CFFEB4747B400F2242B36B10F, vertexName=AAD10383766F45E39B8107B0CA5AC336, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:59,644 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, vertexId=vertex_1463535179407_0001_1_02, initRequestedTime=1463535179626, initedTime=1463535179639, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:32:59,644 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:32:59,678 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:59,678 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:32:59,678 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:59,679 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:59,679 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:59,679 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:32:59,679 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:59,679 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:59,680 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,682 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,684 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,688 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,689 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FFDAAF19BA3149E2BE99E4ACA1A46D82=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:59,690 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FFDAAF19BA3149E2BE99E4ACA1A46D82=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:59,692 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] parallelism set to 1
2016-05-18 01:32:59,692 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,692 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] submitted tez dag to app master: application_1463535179407_0001, with dag id: dag_1463535179407_0001_1
2016-05-18 01:32:59,693 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, vertexId=vertex_1463535179407_0001_1_01, initRequestedTime=1463535179624, initedTime=1463535179693, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:59,693 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] with distanceFromRoot: 0
2016-05-18 01:32:59,693 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:59,692 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] parallelism set to 1
2016-05-18 01:32:59,694 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_01, startRequestedTime=1463535179626, startedTime=1463535179693
2016-05-18 01:32:59,695 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:59,695 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,696 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, vertexId=vertex_1463535179407_0001_1_00, initRequestedTime=1463535179623, initedTime=1463535179695, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:59,696 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] with distanceFromRoot: 0
2016-05-18 01:32:59,696 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:59,696 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:32:59,696 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,696 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535179407_0001_1_01 for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 1 numSources: 2
2016-05-18 01:32:59,697 INFO  impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 1 numSources: 2
2016-05-18 01:32:59,697 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_00, startRequestedTime=1463535179626, startedTime=1463535179697
2016-05-18 01:32:59,697 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:59,697 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,698 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535179407_0001_1_00 for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 2 numSources: 2
2016-05-18 01:32:59,698 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] with distanceFromRoot: 1
2016-05-18 01:32:59,699 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:32:59,699 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,699 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskId=task_1463535179407_0001_1_01_000000, scheduledTime=1463535179698, launchTime=1463535179698
2016-05-18 01:32:59,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_02, startRequestedTime=1463535179698, startedTime=1463535179701
2016-05-18 01:32:59,701 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,704 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskId=task_1463535179407_0001_1_00_000000, scheduledTime=1463535179704, launchTime=1463535179704
2016-05-18 01:32:59,704 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,704 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,706 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:59,720 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: AAD10383766F45E39B8107B0CA5AC336 with 0 source tasks and 1 pending tasks
2016-05-18 01:32:59,723 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B1A06BDE26DF401DB6A2AD73562926CF in vertex: AAD10383766F45E39B8107B0CA5AC336 numBipartiteSourceTasks: 1
2016-05-18 01:32:59,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000001, launchTime=1463535179721
2016-05-18 01:32:59,724 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: BA80601CC7DA4457869BE0F2F8B5EC29 in vertex: AAD10383766F45E39B8107B0CA5AC336 numBipartiteSourceTasks: 2
2016-05-18 01:32:59,726 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000001
2016-05-18 01:32:59,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000002, launchTime=1463535179723
2016-05-18 01:32:59,728 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,728 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_01_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:59,728 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, startTime=1463535179727, containerId=container_1463535179407_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:59,730 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000002
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,731 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,731 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,731 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:59,733 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: B1A06BDE26DF401DB6A2AD73562926CF, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AAD10383766F45E39B8107B0CA5AC336, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:32:59,733 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:59,734 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_00_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:32:59,735 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, startTime=1463535179733, containerId=container_1463535179407_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:32:59,736 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,736 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,738 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535179407_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:32:59,738 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,739 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,739 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,740 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:59,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: BA80601CC7DA4457869BE0F2F8B5EC29, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AAD10383766F45E39B8107B0CA5AC336, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:32:59,740 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:59,741 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,744 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535179407_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:59,745 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:59,755 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:59,777 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:59,783 INFO  input.MRInput (MRInput.java:initialize(440)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:59,783 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,786 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, ordinal: 1
2016-05-18 01:32:59,788 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, mem on start (mb), free: 348, total: 388, max: 672
2016-05-18 01:32:59,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:59,787 INFO  input.MRInput (MRInput.java:initialize(440)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:59,789 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:59,790 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:59,791 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:59,791 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FFDAAF19BA3149E2BE99E4ACA1A46D82:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AAD10383766F45E39B8107B0CA5AC336:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:32:59,792 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:59,792 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:59,794 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:32:59,794 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AAD10383766F45E39B8107B0CA5AC336:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:32:59,794 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:59,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:59,795 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,796 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:59,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:59,797 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:59,801 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,806 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:32:59,808 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 initialized RecordReader from event
2016-05-18 01:32:59,807 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,811 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, ordinal: 0
2016-05-18 01:32:59,811 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, mem on start (mb), free: 346, total: 388, max: 672
2016-05-18 01:32:59,811 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:59,811 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:59,812 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:59,812 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:59,813 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FFDAAF19BA3149E2BE99E4ACA1A46D82:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AAD10383766F45E39B8107B0CA5AC336:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:32:59,813 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:59,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:59,814 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:32:59,815 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AAD10383766F45E39B8107B0CA5AC336:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:32:59,815 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:59,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:59,816 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,817 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:59,817 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:59,818 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:59,818 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,820 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:32:59,821 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 initialized RecordReader from event
2016-05-18 01:32:59,824 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,827 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"] streamed: true, id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,827 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"] streamed: true, id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,827 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']], id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,828 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,827 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']], id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,829 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,875 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AAD10383766F45E39B8107B0CA5AC336 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4a1dde79, comparator=cascading.tuple.hadoop.util.TupleComparator@6ef8fff2, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:32:59,875 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AAD10383766F45E39B8107B0CA5AC336 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4a1dde79, comparator=cascading.tuple.hadoop.util.TupleComparator@1d6ba1c9, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:32:59,881 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false
2016-05-18 01:32:59,881 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false
2016-05-18 01:33:00,224 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:00,225 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:00,225 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:00,228 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AAD10383766F45E39B8107B0CA5AC336: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:00,235 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:33:00,237 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:00,238 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:00,238 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:00,238 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:00,238 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AAD10383766F45E39B8107B0CA5AC336: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:00,240 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:33:00,240 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:00,243 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00,244 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, mem on close (mb), free: 256, total: 475, max: 672
2016-05-18 01:33:00,245 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AAD10383766F45E39B8107B0CA5AC336: Starting flush of map output
2016-05-18 01:33:00,245 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AAD10383766F45E39B8107B0CA5AC336: Span0.length = 5, perItem = 15
2016-05-18 01:33:00,246 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,246 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, mem on close (mb), free: 256, total: 475, max: 672
2016-05-18 01:33:00,247 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AAD10383766F45E39B8107B0CA5AC336: done sorting span=0, length=5, time=1
2016-05-18 01:33:00,248 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AAD10383766F45E39B8107B0CA5AC336: Starting flush of map output
2016-05-18 01:33:00,249 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AAD10383766F45E39B8107B0CA5AC336: Span0.length = 5, perItem = 9
2016-05-18 01:33:00,249 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AAD10383766F45E39B8107B0CA5AC336: done sorting span=0, length=5, time=0
2016-05-18 01:33:00,250 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AAD10383766F45E39B8107B0CA5AC336: Heap = SpanIterator<0:4> (span=Span[16777216,45]),
2016-05-18 01:33:00,250 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AAD10383766F45E39B8107B0CA5AC336: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:00,256 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AAD10383766F45E39B8107B0CA5AC336: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_00_000000_0_10012_0/file.out
2016-05-18 01:33:00,257 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AAD10383766F45E39B8107B0CA5AC336: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_01_000000_0_10010_0/file.out
2016-05-18 01:33:00,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,287 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:00,288 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:00,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:00,288 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:00,291 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:00,291 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, creationTime=1463535179699, allocationTime=1463535179718, startTime=1463535179727, finishTime=1463535180291, timeTaken=564, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,291 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:00,293 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,293 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:00,293 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,292 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:00,293 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,297 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:00,297 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,298 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:00,298 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,298 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000002
2016-05-18 01:33:00,299 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,300 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskId=task_1463535179407_0001_1_01_000000, startTime=1463535179727, finishTime=1463535180299, timeTaken=572, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:00,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, creationTime=1463535179704, allocationTime=1463535179719, startTime=1463535179733, finishTime=1463535180301, timeTaken=568, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,304 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,305 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:33:00,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, vertexId=vertex_1463535179407_0001_1_01, initRequestedTime=1463535179624, initedTime=1463535179693, startRequestedTime=1463535179626, startedTime=1463535179693, finishTime=1463535180305, timeTaken=612, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535179727, firstTasksToStart=[ task_1463535179407_0001_1_01_000000 ], lastTaskFinishTime=1463535180291, lastTasksToFinish=[ task_1463535179407_0001_1_01_000000 ], minTaskDuration=564, maxTaskDuration=564, avgTaskDuration=564.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_01_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,307 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,307 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000001, stoppedTime=1463535180307, exitStatus=0
2016-05-18 01:33:00,308 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00,311 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskId=task_1463535179407_0001_1_00_000000, startTime=1463535179733, finishTime=1463535180308, timeTaken=575, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:00,312 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] attempt: attempt_1463535179407_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:00,312 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,312 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,313 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,313 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000002, stoppedTime=1463535180313, exitStatus=0
2016-05-18 01:33:00,313 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,313 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:33:00,315 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, vertexId=vertex_1463535179407_0001_1_00, initRequestedTime=1463535179623, initedTime=1463535179695, startRequestedTime=1463535179626, startedTime=1463535179697, finishTime=1463535180313, timeTaken=616, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535179733, firstTasksToStart=[ task_1463535179407_0001_1_00_000000 ], lastTaskFinishTime=1463535180301, lastTasksToFinish=[ task_1463535179407_0001_1_00_000000 ], minTaskDuration=568, maxTaskDuration=568, avgTaskDuration=568.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_00_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,315 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,316 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] attempt: attempt_1463535179407_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:00,316 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,316 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,316 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,316 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AAD10383766F45E39B8107B0CA5AC336
2016-05-18 01:33:00,317 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B1A06BDE26DF401DB6A2AD73562926CF, destinationVertexName=AAD10383766F45E39B8107B0CA5AC336. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:00,317 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=BA80601CC7DA4457869BE0F2F8B5EC29, destinationVertexName=AAD10383766F45E39B8107B0CA5AC336. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:00,317 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,318 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskId=task_1463535179407_0001_1_02_000000, scheduledTime=1463535180318, launchTime=1463535180318
2016-05-18 01:33:00,319 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,321 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000003, launchTime=1463535180321
2016-05-18 01:33:00,323 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,324 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,324 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,324 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_02_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:00,324 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000001, Exiting
2016-05-18 01:33:00,324 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,327 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,326 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,326 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, startTime=1463535180324, containerId=container_1463535179407_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:00,332 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,332 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000002: 33 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,332 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000002, Exiting
2016-05-18 01:33:00,332 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000002
2016-05-18 01:33:00 Starting to run new task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,332 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000002
2016-05-18 01:33:00,332 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,332 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,333 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:00,333 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,333 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000001 completed successfully
2016-05-18 01:33:00,334 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,334 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000002 completed successfully
2016-05-18 01:33:00,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: AAD10383766F45E39B8107B0CA5AC336, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1A06BDE26DF401DB6A2AD73562926CF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=BA80601CC7DA4457869BE0F2F8B5EC29, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B5195B2CFFEB4747B400F2242B36B10F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:00,335 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:00,336 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,347 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535179407_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,347 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:00,347 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:00,355 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:00,356 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:00,360 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, ordinal: 2
2016-05-18 01:33:00,360 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, mem on start (mb), free: 244, total: 475, max: 672
2016-05-18 01:33:00,360 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:00,360 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:00,361 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:00,361 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:00,366 INFO  output.MROutput (MROutput.java:initialize(421)) - B5195B2CFFEB4747B400F2242B36B10F: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:00,367 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:00,367 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B5195B2CFFEB4747B400F2242B36B10F:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B1A06BDE26DF401DB6A2AD73562926CF:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [BA80601CC7DA4457869BE0F2F8B5EC29:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:00,367 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:00,367 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:00,367 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:00,368 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B5195B2CFFEB4747B400F2242B36B10F:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B1A06BDE26DF401DB6A2AD73562926CF:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [BA80601CC7DA4457869BE0F2F8B5EC29:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:00,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:00,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B1A06BDE26DF401DB6A2AD73562926CF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:00,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BA80601CC7DA4457869BE0F2F8B5EC29 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:00,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:33:00,368 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:33:00,369 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B1A06BDE26DF401DB6A2AD73562926CF: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:00,374 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B1A06BDE26DF401DB6A2AD73562926CF: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:00,376 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - BA80601CC7DA4457869BE0F2F8B5EC29: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:00,376 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - BA80601CC7DA4457869BE0F2F8B5EC29: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:00,392 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: BA80601CC7DA4457869BE0F2F8B5EC29 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:00,392 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B1A06BDE26DF401DB6A2AD73562926CF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:00,396 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,396 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:00,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:00,398 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,404 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']]
2016-05-18 01:33:00,405 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A06BDE26DF401DB6A2AD73562926CF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:00,405 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BA80601CC7DA4457869BE0F2F8B5EC29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:00,408 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] streamed: true, id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:33:00,408 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"], id: B5195B2CFFEB4747B400F2242B36B10F
2016-05-18 01:33:00,409 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"]
2016-05-18 01:33:00,409 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932, for 2 inputs
2016-05-18 01:33:00,411 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932, for 2 inputs
2016-05-18 01:33:00,414 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535179407_0001_1_00_000000_0_10012} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535180414, TimeTaken=2, Rate=0.05 MB/s
2016-05-18 01:33:00,414 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,414 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535179407_0001_1_01_000000_0_10010} to DISK_DIRECT, csize=65, dsize=61, EndTime=1463535180414, TimeTaken=4, Rate=0.02 MB/s
2016-05-18 01:33:00,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: BA80601CC7DA4457869BE0F2F8B5EC29, wasInterrupted=false
2016-05-18 01:33:00,415 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,416 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,416 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,417 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B1A06BDE26DF401DB6A2AD73562926CF, wasInterrupted=false
2016-05-18 01:33:00,416 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:00,417 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,418 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:00,419 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_01_000000_0_10010/file.out, len=65, isLocal=true
2016-05-18 01:33:00,419 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:00,419 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:00,420 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 65 bytes from disk
2016-05-18 01:33:00,420 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:00,422 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_00_000000_0_10012/file.out, len=95, isLocal=true
2016-05-18 01:33:00,422 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:00,422 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:00,424 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:00,424 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:00,430 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 48 bytes
2016-05-18 01:33:00,430 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:00,430 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,430 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B1A06BDE26DF401DB6A2AD73562926CF: Shuffle Runner thread complete
2016-05-18 01:33:00,430 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, all 2 inputs ready in: 00:00:00.004
2016-05-18 01:33:00,431 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,431 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - BA80601CC7DA4457869BE0F2F8B5EC29: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:00,432 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - BA80601CC7DA4457869BE0F2F8B5EC29: Shuffle Runner thread complete
2016-05-18 01:33:00,435 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B1A06BDE26DF401DB6A2AD73562926CF: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:00,450 INFO  output.MROutput (MROutput.java:close(526)) - B5195B2CFFEB4747B400F2242B36B10F closed
2016-05-18 01:33:00,450 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535179407_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:00,451 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,451 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, mem on close (mb), free: 235, total: 475, max: 672
2016-05-18 01:33:00,452 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,452 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A06BDE26DF401DB6A2AD73562926CF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:00,452 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,452 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BA80601CC7DA4457869BE0F2F8B5EC29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:00,453 INFO  output.MROutput (MROutput.java:close(526)) - B5195B2CFFEB4747B400F2242B36B10F closed
2016-05-18 01:33:00,453 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,454 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:00,454 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:00,455 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:00,456 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:00,456 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,456 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, creationTime=1463535180319, allocationTime=1463535180320, startTime=1463535180324, finishTime=1463535180456, timeTaken=132, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,458 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,456 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:00,458 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,458 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskId=task_1463535179407_0001_1_02_000000, startTime=1463535180324, finishTime=1463535180458, timeTaken=134, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:00,459 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,460 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, vertexId=vertex_1463535179407_0001_1_02, initRequestedTime=1463535179626, initedTime=1463535179639, startRequestedTime=1463535179698, startedTime=1463535179701, finishTime=1463535180460, timeTaken=759, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535180324, firstTasksToStart=[ task_1463535179407_0001_1_02_000000 ], lastTaskFinishTime=1463535180456, lastTasksToFinish=[ task_1463535179407_0001_1_02_000000 ], minTaskDuration=132, maxTaskDuration=132, avgTaskDuration=132.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_02_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,461 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,462 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000003, stoppedTime=1463535180462, exitStatus=0
2016-05-18 01:33:00,462 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,462 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,462 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:33:00,462 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:33:00,462 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535179407_0001_1
2016-05-18 01:33:00,463 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535179407_0001_1
2016-05-18 01:33:00,463 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: B5195B2CFFEB4747B400F2242B36B10F for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], outputName: B5195B2CFFEB4747B400F2242B36B10F
2016-05-18 01:33:00,465 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:00,467 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,469 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,469 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,470 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000003, Exiting
2016-05-18 01:33:00,470 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,470 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,471 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,471 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000003 completed successfully
2016-05-18 01:33:00,476 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:B5195B2CFFEB4747B400F2242B36B10F of vertex/vertexGroup:AAD10383766F45E39B8107B0CA5AC336 isVertexGroupOutput:false
2016-05-18 01:33:00,476 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,479 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535179407_0001_1, queueSize=0
2016-05-18 01:33:00,482 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535179407_0001_1, startTime=1463535179622, finishTime=1463535180476, timeTaken=854, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=2220, AM_GC_TIME_MILLIS=353, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=120, OUTPUT_BYTES_WITH_OVERHEAD=152, OUTPUT_BYTES_PHYSICAL=160, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605540908, Process_Duration=32, Process_End_Time=4390605540940, Read_Duration=10, Tuples_Read=20, Tuples_Written=15, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-18 01:33:00,482 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535179407_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:00,483 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:00 Completed Dag: dag_1463535179407_0001_1
2016-05-18 01:33:00,483 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535179407_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:00,483 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535179407_0001_1_post
2016-05-18 01:33:00,483 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,483 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:00,484 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:00,484 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, with id=dag_1463535179407_0001_1
2016-05-18 01:33:00,484 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, with id=dag_1463535179407_0001_1
2016-05-18 01:33:00,491 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001
2016-05-18 01:33:00,493 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:00,493 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:00,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:00,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:00,494 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:00,494 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:00,494 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:00,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:00,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:00,495 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:00,495 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:00,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:00,496 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:00,499 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:00,504 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001
2016-05-18 01:33:00,505 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit/_temporary
2016-05-18 01:33:00,506 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs]  completed in: 00:01.122
2016-05-18 01:33:00,619 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit
2016-05-18 01:33:00,619 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:33:00,662 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:00,663 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:00,738 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-18 01:33:00,746 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-18 01:33:00,747 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:00,747 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:00,747 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:00,774 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:00,786 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown
2016-05-18 01:33:00,872 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:00,873 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:00,873 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"]
2016-05-18 01:33:00,874 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:00,874 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:00,875 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:00,876 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:00,876 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:00,898 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:00,903 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:00,903 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:00,904 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:00,913 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001 doesn't exist and is created
2016-05-18 01:33:00,934 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:00,934 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:00,935 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd
2016-05-18 01:33:00,951 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535180903_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:00,952 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:00,952 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:00,955 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:00,956 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:00,959 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:00,959 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:00,960 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:00,961 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:00,961 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:00,961 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:00,961 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:00,965 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:00,966 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:00,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:00,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir
2016-05-18 01:33:00,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/history.txt.appattempt_1463535180903_0001_000000, maxErrors=10
2016-05-18 01:33:00,968 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:00,968 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535180903_0001_000000, appSubmitTime=1463535180951, launchTime=1463535180951
2016-05-18 01:33:00,980 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:00,981 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47230
2016-05-18 01:33:00,981 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535180903
2016-05-18 01:33:00,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535180903_0001_000000, startTime=1463535180985
2016-05-18 01:33:00,985 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:01,034 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:01,034 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:01,035 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001, dagName=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,065 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,066 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:01,067 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535180903_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/dag_1463535180903_0001_1.dot
2016-05-18 01:33:01,068 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/dag_1463535180903_0001_1-tez-dag.pb.txt
2016-05-18 01:33:01,466 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535180903_0001_1
2016-05-18 01:33:01,466 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,466 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01 Running Dag: dag_1463535180903_0001_1
2016-05-18 01:33:01,482 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535180903_0001_1, submitTime=1463535181066
2016-05-18 01:33:01,484 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/tez-conf.pb" } size: 40938 timestamp: 1463535180000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:01,484 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,484 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:01,485 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/tez-conf.pb" } size: 40938 timestamp: 1463535180000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:01,485 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,485 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:01,485 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:01,486 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535180903_0001_1, initTime=1463535181485
2016-05-18 01:33:01,486 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:01,489 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:01,489 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_STARTED]: dagID=dag_1463535180903_0001_1, startTime=1463535181489
2016-05-18 01:33:01,489 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:01,489 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001, dagName=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,489 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E654A931A7A841E8A650D30AAFDCC4A1 : {BACEA3E3751449D2B959BDEE247DF8FA={InputName=BACEA3E3751449D2B959BDEE247DF8FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:01,490 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: BACEA3E3751449D2B959BDEE247DF8FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:01,490 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,491 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:01,491 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,492 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: BACEA3E3751449D2B959BDEE247DF8FA on vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,494 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input BACEA3E3751449D2B959BDEE247DF8FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:01,496 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:01,497 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,497 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,497 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535180903_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:01,498 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], numAdditionalOutputs=1
2016-05-18 01:33:01,499 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A59D28D6A5304243ABA39F425135D08A, vertex=vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:01,501 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:01,501 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:01,501 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:01,502 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:01,502 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CAD7E907061C455184DD7CB2ED1DC6A1:A59D28D6A5304243ABA39F425135D08A using oldmapred API
2016-05-18 01:33:01,503 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A59D28D6A5304243ABA39F425135D08A, vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:01,503 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: BACEA3E3751449D2B959BDEE247DF8FA on vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, vertexId=vertex_1463535180903_0001_1_01, initRequestedTime=1463535181491, initedTime=1463535181498, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:01,504 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:01,505 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:01,506 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {BACEA3E3751449D2B959BDEE247DF8FA=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:01,506 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] parallelism set to 1
2016-05-18 01:33:01,506 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:01,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, vertexId=vertex_1463535180903_0001_1_00, initRequestedTime=1463535181489, initedTime=1463535181507, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:01,507 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] with distanceFromRoot: 0
2016-05-18 01:33:01,507 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:01,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535180903_0001_1_00, startRequestedTime=1463535181491, startedTime=1463535181507
2016-05-18 01:33:01,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535180903_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:01,508 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:01,508 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535180903_0001_1_00 for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] numStartedSources: 1 numSources: 1
2016-05-18 01:33:01,508 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] with distanceFromRoot: 1
2016-05-18 01:33:01,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535180903_0001_1_01, startRequestedTime=1463535181508, startedTime=1463535181508
2016-05-18 01:33:01,511 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:01,515 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,515 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,515 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_STARTED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskId=task_1463535180903_0001_1_00_000000, scheduledTime=1463535181515, launchTime=1463535181515
2016-05-18 01:33:01,516 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: CAD7E907061C455184DD7CB2ED1DC6A1 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:01,516 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,519 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E654A931A7A841E8A650D30AAFDCC4A1 in vertex: CAD7E907061C455184DD7CB2ED1DC6A1 numBipartiteSourceTasks: 1
2016-05-18 01:33:01,519 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:01,521 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535180903_0001_00_000001, launchTime=1463535181521
2016-05-18 01:33:01,522 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,524 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,524 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535180903_0001_1_00_000000_0] started. Is using containerId: [container_1463535180903_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:01,525 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, startTime=1463535181524, containerId=container_1463535180903_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,525 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535180903_0001, with dag id: dag_1463535180903_0001_1
2016-05-18 01:33:01,525 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,526 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,526 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:01,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, VertexName: E654A931A7A841E8A650D30AAFDCC4A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535180903_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BACEA3E3751449D2B959BDEE247DF8FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAD7E907061C455184DD7CB2ED1DC6A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:01,527 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:01,527 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,534 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:01,534 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535180903_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:01,535 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535180903_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:01,536 INFO  input.MRInput (MRInput.java:initialize(440)) - BACEA3E3751449D2B959BDEE247DF8FA using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:01,536 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,543 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, ordinal: 0
2016-05-18 01:33:01,543 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, mem on start (mb), free: 360, total: 396, max: 672
2016-05-18 01:33:01,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:01,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:01,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:01,545 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:01,545 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[BACEA3E3751449D2B959BDEE247DF8FA:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CAD7E907061C455184DD7CB2ED1DC6A1:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:01,545 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:01,546 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:01,546 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:01,546 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[BACEA3E3751449D2B959BDEE247DF8FA:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CAD7E907061C455184DD7CB2ED1DC6A1:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:01,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:01,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BACEA3E3751449D2B959BDEE247DF8FA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:01,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:01,548 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:01,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:01,549 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,553 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:01,554 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:01,554 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - BACEA3E3751449D2B959BDEE247DF8FA initialized RecordReader from event
2016-05-18 01:33:01,555 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,556 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{?}:NONE]], id: 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,556 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{?}:NONE]] 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,560 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - CAD7E907061C455184DD7CB2ED1DC6A1 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5c19b253, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@1b0a9662, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:01,560 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for CAD7E907061C455184DD7CB2ED1DC6A1: , UsingHashComparator=false
2016-05-18 01:33:01,572 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:01,573 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:01,573 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for CAD7E907061C455184DD7CB2ED1DC6A1: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:01,573 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - CAD7E907061C455184DD7CB2ED1DC6A1: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:01,574 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:01,574 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:01,577 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,577 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, mem on close (mb), free: 258, total: 396, max: 672
2016-05-18 01:33:01,578 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - CAD7E907061C455184DD7CB2ED1DC6A1: Starting flush of map output
2016-05-18 01:33:01,578 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - CAD7E907061C455184DD7CB2ED1DC6A1: Span0.length = 5, perItem = 27
2016-05-18 01:33:01,578 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - CAD7E907061C455184DD7CB2ED1DC6A1: done sorting span=0, length=5, time=0
2016-05-18 01:33:01,579 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - CAD7E907061C455184DD7CB2ED1DC6A1: Heap = SpanIterator<0:4> (span=Span[16777216,135]),
2016-05-18 01:33:01,580 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - CAD7E907061C455184DD7CB2ED1DC6A1: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-local-dir/output/attempt_1463535180903_0001_1_00_000000_0_10019_0/file.out
2016-05-18 01:33:01,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,585 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:01,585 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535180903_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:01,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535180903_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:01,587 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535180903_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:01,587 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,588 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:01,588 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, creationTime=1463535181516, allocationTime=1463535181520, startTime=1463535181524, finishTime=1463535181587, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:01,588 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,589 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,590 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskId=task_1463535180903_0001_1_00_000000, startTime=1463535181524, finishTime=1463535181590, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463535180903_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:01,591 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:01,591 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, vertexId=vertex_1463535180903_0001_1_00, initRequestedTime=1463535181489, initedTime=1463535181507, startRequestedTime=1463535181491, startedTime=1463535181507, finishTime=1463535181591, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535181524, firstTasksToStart=[ task_1463535180903_0001_1_00_000000 ], lastTaskFinishTime=1463535181587, lastTasksToFinish=[ task_1463535180903_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535180903_0001_1_00_000000 ], longestDurationTasks=[ task_1463535180903_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:01,592 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:01,592 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535180903_0001_00_000001, stoppedTime=1463535181592, exitStatus=0
2016-05-18 01:33:01,593 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] attempt: attempt_1463535180903_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:01,593 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:01,593 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAD7E907061C455184DD7CB2ED1DC6A1
2016-05-18 01:33:01,593 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,594 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E654A931A7A841E8A650D30AAFDCC4A1, destinationVertexName=CAD7E907061C455184DD7CB2ED1DC6A1. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:01,594 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,594 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535180903_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:01,595 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_STARTED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskId=task_1463535180903_0001_1_01_000000, scheduledTime=1463535181595, launchTime=1463535181595
2016-05-18 01:33:01,595 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,597 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535180903_0001_00_000002, launchTime=1463535181597
2016-05-18 01:33:01,598 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,599 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535180903_0001_1_01_000000_0] started. Is using containerId: [container_1463535180903_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:01,599 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, startTime=1463535181599, containerId=container_1463535180903_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:01,600 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,600 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535180903_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:01,601 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:01,601 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535180903_0001_00_000001, Exiting
2016-05-18 01:33:01,601 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,601 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,602 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535180903_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,602 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,602 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,602 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:01,602 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535180903_0001_00_000001 completed successfully
2016-05-18 01:33:01,603 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, VertexName: CAD7E907061C455184DD7CB2ED1DC6A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535180903_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E654A931A7A841E8A650D30AAFDCC4A1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A59D28D6A5304243ABA39F425135D08A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:01,603 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:01,604 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,609 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535180903_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,610 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535180903_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:01,611 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:01,614 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:01,625 INFO  output.MROutput (MROutput.java:initialize(421)) - A59D28D6A5304243ABA39F425135D08A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:01,626 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, ordinal: 1
2016-05-18 01:33:01,626 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, mem on start (mb), free: 253, total: 396, max: 672
2016-05-18 01:33:01,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:01,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:01,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:01,627 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:01,627 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A59D28D6A5304243ABA39F425135D08A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E654A931A7A841E8A650D30AAFDCC4A1:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:01,627 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:01,627 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:01,628 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:01,629 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A59D28D6A5304243ABA39F425135D08A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E654A931A7A841E8A650D30AAFDCC4A1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:01,630 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:01,630 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E654A931A7A841E8A650D30AAFDCC4A1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:01,630 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E654A931A7A841E8A650D30AAFDCC4A1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:01,630 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:01,630 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E654A931A7A841E8A650D30AAFDCC4A1: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:01,631 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:01,632 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E654A931A7A841E8A650D30AAFDCC4A1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:01,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,633 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:01,634 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,636 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E654A931A7A841E8A650D30AAFDCC4A1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:01,638 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{?}:NONE]]
2016-05-18 01:33:01,639 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535180903_0001_1_00_000000_0_10019} to DISK_DIRECT, csize=155, dsize=151, EndTime=1463535181639, TimeTaken=1, Rate=0.15 MB/s
2016-05-18 01:33:01,639 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{?}:NONE]] streamed: true, id: 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,639 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"], id: A59D28D6A5304243ABA39F425135D08A
2016-05-18 01:33:01,639 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,640 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"]
2016-05-18 01:33:01,640 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{?}:NONE]] 0D0222E4052A464EBAE45B11F76EE997, for 1 inputs
2016-05-18 01:33:01,640 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:01,641 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E654A931A7A841E8A650D30AAFDCC4A1, wasInterrupted=false
2016-05-18 01:33:01,641 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:01,643 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:01,644 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-local-dir/output/attempt_1463535180903_0001_1_00_000000_0_10019/file.out, len=155, isLocal=true
2016-05-18 01:33:01,644 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 155 bytes from disk
2016-05-18 01:33:01,644 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:01,645 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:01,645 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 137 bytes
2016-05-18 01:33:01,645 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,645 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E654A931A7A841E8A650D30AAFDCC4A1: Shuffle Runner thread complete
2016-05-18 01:33:01,645 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, all 1 inputs ready in: 00:00:00.005
2016-05-18 01:33:01,646 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E654A931A7A841E8A650D30AAFDCC4A1: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:01,654 INFO  output.MROutput (MROutput.java:close(526)) - A59D28D6A5304243ABA39F425135D08A closed
2016-05-18 01:33:01,655 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535180903_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:01,656 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,656 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, mem on close (mb), free: 250, total: 396, max: 672
2016-05-18 01:33:01,656 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,656 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E654A931A7A841E8A650D30AAFDCC4A1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:01,657 INFO  output.MROutput (MROutput.java:close(526)) - A59D28D6A5304243ABA39F425135D08A closed
2016-05-18 01:33:01,657 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,658 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:01,658 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535180903_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:01,659 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535180903_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:01,660 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535180903_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:01,661 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,661 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, creationTime=1463535181595, allocationTime=1463535181597, startTime=1463535181599, finishTime=1463535181661, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:01,661 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:01,662 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,662 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,662 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,663 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskId=task_1463535180903_0001_1_01_000000, startTime=1463535181599, finishTime=1463535181662, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463535180903_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:01,664 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:01,664 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, vertexId=vertex_1463535180903_0001_1_01, initRequestedTime=1463535181491, initedTime=1463535181498, startRequestedTime=1463535181508, startedTime=1463535181508, finishTime=1463535181664, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535181599, firstTasksToStart=[ task_1463535180903_0001_1_01_000000 ], lastTaskFinishTime=1463535181661, lastTasksToFinish=[ task_1463535180903_0001_1_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535180903_0001_1_01_000000 ], longestDurationTasks=[ task_1463535180903_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:01,665 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:01,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535180903_0001_00_000002, stoppedTime=1463535181665, exitStatus=0
2016-05-18 01:33:01,665 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:01,666 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,666 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,666 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535180903_0001_1
2016-05-18 01:33:01,666 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535180903_0001_1
2016-05-18 01:33:01,666 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A59D28D6A5304243ABA39F425135D08A for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], outputName: A59D28D6A5304243ABA39F425135D08A
2016-05-18 01:33:01,666 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:01,667 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535180903_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:01,673 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535180903_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:01,674 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:01,674 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535180903_0001_00_000002, Exiting
2016-05-18 01:33:01,674 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,674 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,674 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535180903_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:01,674 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535180903_0001_00_000002 completed successfully
2016-05-18 01:33:01,677 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A59D28D6A5304243ABA39F425135D08A of vertex/vertexGroup:CAD7E907061C455184DD7CB2ED1DC6A1 isVertexGroupOutput:false
2016-05-18 01:33:01,678 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,680 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535180903_0001_1, queueSize=0
2016-05-18 01:33:01,681 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535180903_0001_1, startTime=1463535181489, finishTime=1463535181678, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=320, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070363220, Process_Duration=13, Process_End_Time=2927070363233, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:01,681 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535180903_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:01,681 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:01 Completed Dag: dag_1463535180903_0001_1
2016-05-18 01:33:01,681 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535180903_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:01,682 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535180903_0001_1_post
2016-05-18 01:33:01,682 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,682 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:01,682 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:01,682 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, with id=dag_1463535180903_0001_1
2016-05-18 01:33:01,682 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, with id=dag_1463535180903_0001_1
2016-05-18 01:33:01,695 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001
2016-05-18 01:33:01,697 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:01,697 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:01,698 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:01,698 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:01,698 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:01,698 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:01,698 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:01,699 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:01,699 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:01,699 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:01,699 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:01,700 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:01,700 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:01,702 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:01,705 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001
2016-05-18 01:33:01,705 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown/_temporary
2016-05-18 01:33:01,706 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.833
2016-05-18 01:33:01,748 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown
2016-05-18 01:33:01,780 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:01,780 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:01,831 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:01,845 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 01:33:01,845 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:01,846 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:01,846 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:01,867 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:33:01,868 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre
2016-05-18 01:33:01,884 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:01,884 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:01,885 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"]
2016-05-18 01:33:01,885 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:01,885 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:01,885 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:01,886 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:01,886 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:01,906 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:01,910 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:01,911 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:01,911 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:01,920 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001 doesn't exist and is created
2016-05-18 01:33:01,941 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:01,941 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:01,942 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd
2016-05-18 01:33:01,958 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535181911_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:01,958 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:01,959 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:01,960 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:01,961 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:01,963 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:01,963 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:01,964 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:01,964 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:01,965 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:01,965 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:01,965 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:01,968 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:01,969 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:01,969 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:01,969 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir
2016-05-18 01:33:01,970 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/history.txt.appattempt_1463535181911_0001_000000, maxErrors=10
2016-05-18 01:33:01,970 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:01,971 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535181911_0001_000000, appSubmitTime=1463535181957, launchTime=1463535181958
2016-05-18 01:33:01,979 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:01,987 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34625
2016-05-18 01:33:01,987 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535181911
2016-05-18 01:33:01,990 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535181911_0001_000000, startTime=1463535181989
2016-05-18 01:33:01,990 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,041 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,041 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,042 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001, dagName=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,068 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,068 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:02,069 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535181911_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/dag_1463535181911_0001_1.dot
2016-05-18 01:33:02,070 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/dag_1463535181911_0001_1-tez-dag.pb.txt
2016-05-18 01:33:02,075 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535181911_0001_1
2016-05-18 01:33:02,076 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,076 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02 Running Dag: dag_1463535181911_0001_1
2016-05-18 01:33:02,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535181911_0001_1, submitTime=1463535182068
2016-05-18 01:33:02,092 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001/tez-conf.pb" } size: 40938 timestamp: 1463535181000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:02,092 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,092 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,092 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:02,093 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:02,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535181911_0001_1, initTime=1463535182093
2016-05-18 01:33:02,094 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:02,094 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:02,095 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_STARTED]: dagID=dag_1463535181911_0001_1, startTime=1463535182095
2016-05-18 01:33:02,095 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:02,095 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F9FB7ED0624B47BC882DF714B6949DC1 : {1FE6BD0DC88D434898269C5A9D551BA7={InputName=1FE6BD0DC88D434898269C5A9D551BA7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:02,096 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1FE6BD0DC88D434898269C5A9D551BA7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:02,096 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,097 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001, dagName=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,097 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:02,097 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1FE6BD0DC88D434898269C5A9D551BA7 on vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,101 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1FE6BD0DC88D434898269C5A9D551BA7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:02,104 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:02,104 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:33:02,105 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:02,105 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:02,105 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1FE6BD0DC88D434898269C5A9D551BA7 on vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,106 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,107 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1FE6BD0DC88D434898269C5A9D551BA7=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:02,108 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] parallelism set to 1
2016-05-18 01:33:02,108 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,108 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], numAdditionalOutputs=1
2016-05-18 01:33:02,108 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=083C7AB3509047089BD4A6838B62AC11, vertex=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:02,111 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F9FB7ED0624B47BC882DF714B6949DC1:083C7AB3509047089BD4A6838B62AC11 using oldmapred API
2016-05-18 01:33:02,112 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=083C7AB3509047089BD4A6838B62AC11, vertexName=F9FB7ED0624B47BC882DF714B6949DC1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:02,115 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, initRequestedTime=1463535182095, initedTime=1463535182108, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:02,115 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] with distanceFromRoot: 0
2016-05-18 01:33:02,115 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:02,116 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535181911_0001_1_00, startRequestedTime=1463535182097, startedTime=1463535182115
2016-05-18 01:33:02,117 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535181911_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:02,117 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:02,117 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F9FB7ED0624B47BC882DF714B6949DC1
2016-05-18 01:33:02,117 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,118 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_STARTED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskId=task_1463535181911_0001_1_00_000000, scheduledTime=1463535182118, launchTime=1463535182118
2016-05-18 01:33:02,119 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,120 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:02,122 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535181911_0001_00_000001, launchTime=1463535182122
2016-05-18 01:33:02,123 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,124 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535181911_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,124 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535181911_0001_1_00_000000_0] started. Is using containerId: [container_1463535181911_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,124 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0, startTime=1463535182124, containerId=container_1463535181911_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:02,125 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,125 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,125 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:02,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, VertexName: F9FB7ED0624B47BC882DF714B6949DC1, VertexParallelism: 1, TaskAttemptID:attempt_1463535181911_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1FE6BD0DC88D434898269C5A9D551BA7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=083C7AB3509047089BD4A6838B62AC11, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:02,125 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:02,126 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,127 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535181911_0001, with dag id: dag_1463535181911_0001_1
2016-05-18 01:33:02,133 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535181911_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:02,133 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535181911_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:02,133 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:02,137 INFO  input.MRInput (MRInput.java:initialize(440)) - 1FE6BD0DC88D434898269C5A9D551BA7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:02,138 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,146 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, ordinal: 0
2016-05-18 01:33:02,146 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, mem on start (mb), free: 173, total: 396, max: 672
2016-05-18 01:33:02,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:02,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:02,147 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:02,150 INFO  output.MROutput (MROutput.java:initialize(421)) - 083C7AB3509047089BD4A6838B62AC11: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:02,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:02,150 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1FE6BD0DC88D434898269C5A9D551BA7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [083C7AB3509047089BD4A6838B62AC11:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:02,151 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:02,151 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:02,151 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:02,151 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1FE6BD0DC88D434898269C5A9D551BA7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [083C7AB3509047089BD4A6838B62AC11:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:02,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:02,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1FE6BD0DC88D434898269C5A9D551BA7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:02,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:02,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:02,154 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:02,154 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,157 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:33:02,157 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 1FE6BD0DC88D434898269C5A9D551BA7 initialized RecordReader from event
2016-05-18 01:33:02,158 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:02,158 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,158 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"], id: 083C7AB3509047089BD4A6838B62AC11
2016-05-18 01:33:02,159 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"]
2016-05-18 01:33:02,159 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:02,159 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:02,161 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught OutOfMemoryException, will not trap, rethrowing
java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
	at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
	at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:02,163 INFO  output.MROutput (MROutput.java:close(526)) - 083C7AB3509047089BD4A6838B62AC11 closed
2016-05-18 01:33:02,164 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535181911_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:02,166 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535181911_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:02,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535181911_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463535182159, Process_Duration=6, Process_End_Time=1463535182165, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 01:33:02,166 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 01:33:02,167 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, mem on close (mb), free: 171, total: 396, max: 672
2016-05-18 01:33:02,167 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=F9FB7ED0624B47BC882DF714B6949DC1, index=0, interruptedStatus=false
2016-05-18 01:33:02,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=F9FB7ED0624B47BC882DF714B6949DC1, sourceVertex=1FE6BD0DC88D434898269C5A9D551BA7, interruptedStatus=true
2016-05-18 01:33:02,168 INFO  output.MROutput (MROutput.java:close(526)) - 083C7AB3509047089BD4A6838B62AC11 closed
2016-05-18 01:33:02,168 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=F9FB7ED0624B47BC882DF714B6949DC1, sourceVertex=083C7AB3509047089BD4A6838B62AC11, interruptedStatus=true
2016-05-18 01:33:02,172 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535181911_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}  
2016-05-18 01:33:02,173 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:02,173 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,173 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,174 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535181911_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=java.lang.OutOfMemoryError: fake error, errorMessage='TaskExecutionFailure: fake error'}
2016-05-18 01:33:02,174 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463535181911_0001_00_000001 completed but with errors
2016-05-18 01:33:02,175 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0, creationTime=1463535182119, allocationTime=1463535182121, startTime=1463535182124, finishTime=1463535182172, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
	at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
	at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535182159, Process_Duration=6, Process_End_Time=1463535182165, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 01:33:02,178 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463535181911_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: fake error
2016-05-18 01:33:02,178 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535181911_0001_00_000001, stoppedTime=1463535182178, exitStatus=1
2016-05-18 01:33:02,178 INFO  impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463535181911_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 01:33:02,179 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskId=task_1463535181911_0001_1_00_000000, startTime=1463535182124, finishTime=1463535182179, timeTaken=55, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
	at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
	at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-18 01:33:02,181 INFO  impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] because task failed: task_1463535181911_0001_1_00_000000
2016-05-18 01:33:02,182 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] due to trigger: OWN_TASK_FAILURE
2016-05-18 01:33:02,183 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 01:33:02,183 INFO  impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 01:33:02,183 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,184 INFO  impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=083C7AB3509047089BD4A6838B62AC11, vertexId=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,188 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, initRequestedTime=1463535182095, initedTime=1463535182108, startRequestedTime=1463535182097, startedTime=1463535182115, finishTime=1463535182187, timeTaken=72, status=FAILED, diagnostics=Task failed, taskId=task_1463535181911_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
	at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
	at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463535182124, firstTasksToStart=[ task_1463535181911_0001_1_00_000000 ], lastTaskFinishTime=1463535182172, lastTasksToFinish=[ task_1463535181911_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 01:33:02,189 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 01:33:02,189 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463535181911_0001_1_00_000000_0 failed: true container: container_1463535181911_0001_00_000001 numFailedTAs: 0
2016-05-18 01:33:02,191 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 01:33:02,191 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 01:33:02,191 INFO  impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 01:33:02,192 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535181911_0001_1, queueSize=0
2016-05-18 01:33:02,192 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535181911_0001_1, startTime=1463535182094, finishTime=1463535182192, timeTaken=98, status=FAILED, diagnostics=Vertex failed, vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, diagnostics=[Task failed, taskId=task_1463535181911_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
	at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
	at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0
2016-05-18 01:33:02,193 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,193 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535181911_0001_1 finished with state: FAILED
2016-05-18 01:33:02,193 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:02 Completed Dag: dag_1463535181911_0001_1
2016-05-18 01:33:02,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535181911_0001_1, dagState=FAILED
2016-05-18 01:33:02,194 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535181911_0001_1_post
2016-05-18 01:33:02,194 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:02,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:02,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, with id=dag_1463535181911_0001_1
2016-05-18 01:33:02,194 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, with id=dag_1463535181911_0001_1
2016-05-18 01:33:02,206 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001
2016-05-18 01:33:02,210 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:02,210 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:02,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:02,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:02,210 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:02,211 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:02,211 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:02,212 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:02,212 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:02,212 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,213 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:02,213 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:02,214 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,214 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:02,218 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001
2016-05-18 01:33:02,218 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-18 01:33:02,218 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,220 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-18 01:33:02,220 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre/_temporary
2016-05-18 01:33:02,220 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.336
exception = cascading.flow.FlowException: local step failed: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,221 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre
2016-05-18 01:33:02,262 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:02,262 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:02,309 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 01:33:02,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:02,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,312 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:02,330 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:02,330 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy
2016-05-18 01:33:02,340 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:02,340 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,342 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"]
2016-05-18 01:33:02,342 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:02,342 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:02,342 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:02,342 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,343 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:02,362 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:02,367 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,367 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:02,367 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:02,376 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001 doesn't exist and is created
2016-05-18 01:33:02,396 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:02,396 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:02,397 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd
2016-05-18 01:33:02,413 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535182367_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,414 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,414 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,416 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:02,418 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:02,420 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:02,420 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:02,420 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:02,421 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:02,421 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:02,421 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:02,421 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:02,422 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:02,422 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,422 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,422 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:02,423 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:02,425 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:02,425 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:02,426 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:02,426 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir
2016-05-18 01:33:02,426 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/history.txt.appattempt_1463535182367_0001_000000, maxErrors=10
2016-05-18 01:33:02,427 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:02,427 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535182367_0001_000000, appSubmitTime=1463535182413, launchTime=1463535182413
2016-05-18 01:33:02,433 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60993
2016-05-18 01:33:02,437 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535182367
2016-05-18 01:33:02,435 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:02,438 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535182367_0001_000000, startTime=1463535182438
2016-05-18 01:33:02,439 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,496 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,497 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,497 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001, dagName=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,516 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,516 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:02,517 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535182367_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/dag_1463535182367_0001_1.dot
2016-05-18 01:33:02,518 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/dag_1463535182367_0001_1-tez-dag.pb.txt
2016-05-18 01:33:02,524 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182367_0001_1
2016-05-18 01:33:02,524 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,524 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02 Running Dag: dag_1463535182367_0001_1
2016-05-18 01:33:02,539 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535182367_0001_1, submitTime=1463535182516
2016-05-18 01:33:02,541 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001/tez-conf.pb" } size: 40938 timestamp: 1463535182000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:02,541 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,542 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,542 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:02,542 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:02,544 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535182367_0001_1, initTime=1463535182542
2016-05-18 01:33:02,544 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:02,544 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:02,545 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_STARTED]: dagID=dag_1463535182367_0001_1, startTime=1463535182544
2016-05-18 01:33:02,545 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001, dagName=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,545 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:02,545 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BDFF45C31B1C4FC48B7F47203B906A87 : {E81CD38A86FB4C398F525C74135279CC={InputName=E81CD38A86FB4C398F525C74135279CC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:02,545 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E81CD38A86FB4C398F525C74135279CC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:02,545 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:02,547 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E81CD38A86FB4C398F525C74135279CC on vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,549 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E81CD38A86FB4C398F525C74135279CC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:02,555 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:02,555 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:02,555 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:02,555 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:02,555 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E81CD38A86FB4C398F525C74135279CC on vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,557 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,557 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E81CD38A86FB4C398F525C74135279CC=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:02,558 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] parallelism set to 1
2016-05-18 01:33:02,560 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,560 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], numAdditionalOutputs=1
2016-05-18 01:33:02,560 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=1A09E352C89E442FAC701DD3B4FB481A, vertex=vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:02,563 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BDFF45C31B1C4FC48B7F47203B906A87:1A09E352C89E442FAC701DD3B4FB481A using oldmapred API
2016-05-18 01:33:02,563 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=1A09E352C89E442FAC701DD3B4FB481A, vertexName=BDFF45C31B1C4FC48B7F47203B906A87, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:02,565 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, vertexId=vertex_1463535182367_0001_1_00, initRequestedTime=1463535182545, initedTime=1463535182560, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:02,565 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] with distanceFromRoot: 0
2016-05-18 01:33:02,567 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:02,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535182367_0001_1_00, startRequestedTime=1463535182546, startedTime=1463535182567
2016-05-18 01:33:02,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535182367_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:02,568 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:02,569 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BDFF45C31B1C4FC48B7F47203B906A87
2016-05-18 01:33:02,569 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,569 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_STARTED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskId=task_1463535182367_0001_1_00_000000, scheduledTime=1463535182569, launchTime=1463535182569
2016-05-18 01:33:02,570 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,571 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:02,573 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535182367_0001_00_000001, launchTime=1463535182573
2016-05-18 01:33:02,574 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,575 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182367_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,575 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535182367_0001_1_00_000000_0] started. Is using containerId: [container_1463535182367_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:02,576 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, startTime=1463535182575, containerId=container_1463535182367_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,576 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,577 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,576 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535182367_0001, with dag id: dag_1463535182367_0001_1
2016-05-18 01:33:02,577 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:02,578 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, VertexName: BDFF45C31B1C4FC48B7F47203B906A87, VertexParallelism: 1, TaskAttemptID:attempt_1463535182367_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E81CD38A86FB4C398F525C74135279CC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1A09E352C89E442FAC701DD3B4FB481A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:02,579 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:02,579 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,584 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:02,586 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535182367_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:02,588 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535182367_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:02,591 INFO  input.MRInput (MRInput.java:initialize(440)) - E81CD38A86FB4C398F525C74135279CC using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:02,591 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,597 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, ordinal: 0
2016-05-18 01:33:02,597 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, mem on start (mb), free: 312, total: 394, max: 672
2016-05-18 01:33:02,597 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:02,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:02,598 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:02,604 INFO  output.MROutput (MROutput.java:initialize(421)) - 1A09E352C89E442FAC701DD3B4FB481A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:02,604 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:02,604 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E81CD38A86FB4C398F525C74135279CC:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [1A09E352C89E442FAC701DD3B4FB481A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:02,604 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:02,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:02,605 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:02,605 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E81CD38A86FB4C398F525C74135279CC:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [1A09E352C89E442FAC701DD3B4FB481A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:02,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:02,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E81CD38A86FB4C398F525C74135279CC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:02,606 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:02,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:02,607 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:02,608 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,609 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:02,609 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - E81CD38A86FB4C398F525C74135279CC initialized RecordReader from event
2016-05-18 01:33:02,612 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,613 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,613 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"], id: 1A09E352C89E442FAC701DD3B4FB481A
2016-05-18 01:33:02,613 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"]
2016-05-18 01:33:02,614 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,614 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:02,616 INFO  output.MROutput (MROutput.java:close(526)) - 1A09E352C89E442FAC701DD3B4FB481A closed
2016-05-18 01:33:02,617 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535182367_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:02,618 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,618 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, mem on close (mb), free: 310, total: 394, max: 672
2016-05-18 01:33:02,621 INFO  output.MROutput (MROutput.java:close(526)) - 1A09E352C89E442FAC701DD3B4FB481A closed
2016-05-18 01:33:02,621 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:02 Completed running task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,623 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:02,623 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535182367_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:02,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535182367_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:02,627 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535182367_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:02,627 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:02,627 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:02,627 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, creationTime=1463535182570, allocationTime=1463535182572, startTime=1463535182575, finishTime=1463535182627, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:02,628 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,628 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,628 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,629 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskId=task_1463535182367_0001_1_00_000000, startTime=1463535182575, finishTime=1463535182628, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463535182367_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:02,629 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:02,629 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,631 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, vertexId=vertex_1463535182367_0001_1_00, initRequestedTime=1463535182545, initedTime=1463535182560, startRequestedTime=1463535182546, startedTime=1463535182567, finishTime=1463535182630, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535182575, firstTasksToStart=[ task_1463535182367_0001_1_00_000000 ], lastTaskFinishTime=1463535182627, lastTasksToFinish=[ task_1463535182367_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535182367_0001_1_00_000000 ], longestDurationTasks=[ task_1463535182367_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:02,631 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:02,632 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535182367_0001_00_000001, stoppedTime=1463535182632, exitStatus=0
2016-05-18 01:33:02,632 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:02,632 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:02,633 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535182367_0001_1
2016-05-18 01:33:02,633 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535182367_0001_1
2016-05-18 01:33:02,633 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 1A09E352C89E442FAC701DD3B4FB481A for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], outputName: 1A09E352C89E442FAC701DD3B4FB481A
2016-05-18 01:33:02,635 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:02,635 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535182367_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:02,639 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535182367_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:02,639 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182367_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:02,639 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535182367_0001_00_000001, Exiting
2016-05-18 01:33:02,639 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,639 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,640 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535182367_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:02,641 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535182367_0001_00_000001 completed successfully
2016-05-18 01:33:02,644 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:1A09E352C89E442FAC701DD3B4FB481A of vertex/vertexGroup:BDFF45C31B1C4FC48B7F47203B906A87 isVertexGroupOutput:false
2016-05-18 01:33:02,644 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:02,646 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535182367_0001_1, queueSize=0
2016-05-18 01:33:02,646 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535182367_0001_1, startTime=1463535182544, finishTime=1463535182644, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:02,646 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535182367_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:02,646 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:02 Completed Dag: dag_1463535182367_0001_1
2016-05-18 01:33:02,647 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535182367_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:02,647 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182367_0001_1_post
2016-05-18 01:33:02,647 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,647 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:02,647 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:02,648 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, with id=dag_1463535182367_0001_1
2016-05-18 01:33:02,648 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, with id=dag_1463535182367_0001_1
2016-05-18 01:33:02,654 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001
2016-05-18 01:33:02,657 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:02,657 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:02,657 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:02,657 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:02,657 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:02,657 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:02,658 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:02,658 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:02,658 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:02,658 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,658 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:02,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:02,659 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,659 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:02,663 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001
2016-05-18 01:33:02,663 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy/_temporary
2016-05-18 01:33:02,664 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.323
2016-05-18 01:33:02,702 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCopy/copy
2016-05-18 01:33:02,735 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:02,735 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:02,782 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 01:33:02,796 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 01:33:02,796 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,797 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,797 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:02,824 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:02,825 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace
2016-05-18 01:33:02,838 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:02,838 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,840 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"]
2016-05-18 01:33:02,840 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:02,840 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:02,840 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:02,841 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:02,841 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:02,856 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:02,860 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,861 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:02,861 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:02,872 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001 doesn't exist and is created
2016-05-18 01:33:02,893 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:02,893 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:02,894 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd
2016-05-18 01:33:02,910 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535182861_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,911 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,911 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,913 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:02,913 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:02,913 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:02,914 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:02,915 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:02,916 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:02,917 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:02,917 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:02,917 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:02,917 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:02,918 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:02,918 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:02,918 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:02,918 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,919 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,919 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:02,919 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:02,922 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:02,922 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:02,923 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:02,923 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir
2016-05-18 01:33:02,924 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/history.txt.appattempt_1463535182861_0001_000000, maxErrors=10
2016-05-18 01:33:02,924 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:02,924 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535182861_0001_000000, appSubmitTime=1463535182909, launchTime=1463535182910
2016-05-18 01:33:02,930 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:02,934 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57199
2016-05-18 01:33:02,935 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535182861
2016-05-18 01:33:02,937 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535182861_0001_000000, startTime=1463535182937
2016-05-18 01:33:02,937 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,993 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,994 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,994 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001, dagName=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,013 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,014 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:03,015 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535182861_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/dag_1463535182861_0001_1.dot
2016-05-18 01:33:03,015 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/dag_1463535182861_0001_1-tez-dag.pb.txt
2016-05-18 01:33:03,023 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182861_0001_1
2016-05-18 01:33:03,023 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,023 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03 Running Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03,038 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535182861_0001_1, submitTime=1463535183013
2016-05-18 01:33:03,039 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001/tez-conf.pb" } size: 40982 timestamp: 1463535182000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:03,040 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,040 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,040 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:03,040 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:03,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535182861_0001_1, initTime=1463535183040
2016-05-18 01:33:03,042 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:03,043 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:03,043 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_STARTED]: dagID=dag_1463535182861_0001_1, startTime=1463535183043
2016-05-18 01:33:03,044 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:03,043 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001, dagName=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FA8E2E97BCD24234B2E4B22352791A45 : {26388EC893B14C818417AC578F943D16={InputName=26388EC893B14C818417AC578F943D16}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 26388EC893B14C818417AC578F943D16, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,045 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:03,045 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 26388EC893B14C818417AC578F943D16 on vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,047 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 26388EC893B14C818417AC578F943D16 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:03,049 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:03,049 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:03,050 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:03,050 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:03,050 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 26388EC893B14C818417AC578F943D16 on vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,052 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,052 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {26388EC893B14C818417AC578F943D16=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:03,053 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] parallelism set to 1
2016-05-18 01:33:03,053 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,053 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], numAdditionalOutputs=1
2016-05-18 01:33:03,053 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=9B3EA2D747BC48BCACA9F2B3E249EBC1, vertex=vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:03,056 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FA8E2E97BCD24234B2E4B22352791A45:9B3EA2D747BC48BCACA9F2B3E249EBC1 using oldmapred API
2016-05-18 01:33:03,056 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=9B3EA2D747BC48BCACA9F2B3E249EBC1, vertexName=FA8E2E97BCD24234B2E4B22352791A45, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:03,057 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, vertexId=vertex_1463535182861_0001_1_00, initRequestedTime=1463535183044, initedTime=1463535183053, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:03,058 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] with distanceFromRoot: 0
2016-05-18 01:33:03,058 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:03,058 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535182861_0001_1_00, startRequestedTime=1463535183045, startedTime=1463535183058
2016-05-18 01:33:03,059 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535182861_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:03,059 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:03,059 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FA8E2E97BCD24234B2E4B22352791A45
2016-05-18 01:33:03,059 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,060 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_STARTED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskId=task_1463535182861_0001_1_00_000000, scheduledTime=1463535183060, launchTime=1463535183060
2016-05-18 01:33:03,060 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,061 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:03,063 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535182861_0001_00_000001, launchTime=1463535183063
2016-05-18 01:33:03,064 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,065 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535182861_0001_1_00_000000_0] started. Is using containerId: [container_1463535182861_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:03,065 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, startTime=1463535183065, containerId=container_1463535182861_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:03,066 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535182861_0001, with dag id: dag_1463535182861_0001_1
2016-05-18 01:33:03,066 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182861_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,067 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,067 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,069 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:03,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, VertexName: FA8E2E97BCD24234B2E4B22352791A45, VertexParallelism: 1, TaskAttemptID:attempt_1463535182861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=26388EC893B14C818417AC578F943D16, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9B3EA2D747BC48BCACA9F2B3E249EBC1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:03,070 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:03,071 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,074 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535182861_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:03,075 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535182861_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:03,080 INFO  input.MRInput (MRInput.java:initialize(440)) - 26388EC893B14C818417AC578F943D16 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:03,080 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,080 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:03,089 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, ordinal: 0
2016-05-18 01:33:03,089 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, mem on start (mb), free: 232, total: 394, max: 672
2016-05-18 01:33:03,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:03,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:03,089 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:03,094 INFO  output.MROutput (MROutput.java:initialize(421)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:03,094 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:03,094 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[26388EC893B14C818417AC578F943D16:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [9B3EA2D747BC48BCACA9F2B3E249EBC1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:03,094 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:03,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:03,095 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:03,095 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[26388EC893B14C818417AC578F943D16:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [9B3EA2D747BC48BCACA9F2B3E249EBC1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:03,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:03,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 26388EC893B14C818417AC578F943D16 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:03,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:03,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:03,095 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,096 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:03,096 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,099 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:03,099 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 26388EC893B14C818417AC578F943D16 initialized RecordReader from event
2016-05-18 01:33:03,100 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:03,101 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,101 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"], id: 9B3EA2D747BC48BCACA9F2B3E249EBC1
2016-05-18 01:33:03,101 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"]
2016-05-18 01:33:03,102 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:03,102 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:03,104 INFO  output.MROutput (MROutput.java:close(526)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1 closed
2016-05-18 01:33:03,105 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535182861_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:03,106 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,106 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, mem on close (mb), free: 230, total: 394, max: 672
2016-05-18 01:33:03,106 INFO  output.MROutput (MROutput.java:close(526)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1 closed
2016-05-18 01:33:03,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,108 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:03,108 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535182861_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:03,108 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535182861_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:03,109 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535182861_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:03,109 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,110 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:03,110 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,110 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,110 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, creationTime=1463535183060, allocationTime=1463535183062, startTime=1463535183065, finishTime=1463535183110, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:03,111 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskId=task_1463535182861_0001_1_00_000000, startTime=1463535183065, finishTime=1463535183112, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463535182861_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:03,113 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:03,113 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,115 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, vertexId=vertex_1463535182861_0001_1_00, initRequestedTime=1463535183044, initedTime=1463535183053, startRequestedTime=1463535183045, startedTime=1463535183058, finishTime=1463535183114, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535183065, firstTasksToStart=[ task_1463535182861_0001_1_00_000000 ], lastTaskFinishTime=1463535183110, lastTasksToFinish=[ task_1463535182861_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535182861_0001_1_00_000000 ], longestDurationTasks=[ task_1463535182861_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:03,115 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:03,115 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535182861_0001_00_000001, stoppedTime=1463535183115, exitStatus=0
2016-05-18 01:33:03,116 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:03,116 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,116 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535182861_0001_1
2016-05-18 01:33:03,117 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535182861_0001_1
2016-05-18 01:33:03,117 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:03,117 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 9B3EA2D747BC48BCACA9F2B3E249EBC1 for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], outputName: 9B3EA2D747BC48BCACA9F2B3E249EBC1
2016-05-18 01:33:03,117 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535182861_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:03,121 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535182861_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:03,121 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182861_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:03,121 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535182861_0001_00_000001, Exiting
2016-05-18 01:33:03,121 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,121 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,123 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535182861_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:03,123 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535182861_0001_00_000001 completed successfully
2016-05-18 01:33:03,129 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:9B3EA2D747BC48BCACA9F2B3E249EBC1 of vertex/vertexGroup:FA8E2E97BCD24234B2E4B22352791A45 isVertexGroupOutput:false
2016-05-18 01:33:03,129 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,130 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535182861_0001_1, queueSize=0
2016-05-18 01:33:03,131 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535182861_0001_1, startTime=1463535183043, finishTime=1463535183129, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:03,131 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535182861_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:03,131 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:03 Completed Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03,132 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535182861_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:03,132 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182861_0001_1_post
2016-05-18 01:33:03,132 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,132 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:03,132 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:03,132 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, with id=dag_1463535182861_0001_1
2016-05-18 01:33:03,132 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, with id=dag_1463535182861_0001_1
2016-05-18 01:33:03,145 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001
2016-05-18 01:33:03,148 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:03,148 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:03,148 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:03,149 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:03,149 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:03,149 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:03,150 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:03,150 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:03,150 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:03,150 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,151 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:03,151 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:03,152 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,153 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:03,156 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001
2016-05-18 01:33:03,157 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace/_temporary
2016-05-18 01:33:03,157 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.319
2016-05-18 01:33:03,201 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:03,202 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:03,237 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace
2016-05-18 01:33:03,268 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:03,268 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:03,311 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 01:33:03,329 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 01:33:03,331 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,331 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,331 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:03,352 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:33:03,353 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand
2016-05-18 01:33:03,366 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:03,367 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,367 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"]
2016-05-18 01:33:03,367 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:03,369 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:03,369 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:03,370 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...egression/complexlogicand
2016-05-18 01:33:03,370 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:03,393 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:03,398 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,398 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:03,399 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:03,409 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001 doesn't exist and is created
2016-05-18 01:33:03,429 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:03,429 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:03,430 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd
2016-05-18 01:33:03,446 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535183398_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,446 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,446 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,448 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:03,448 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:03,449 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:03,449 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:03,450 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:03,452 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:03,452 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:03,453 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:03,453 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:03,454 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:03,454 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:03,454 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:03,455 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:03,455 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,455 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,456 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:03,456 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:03,458 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:03,459 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:03,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:03,460 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir
2016-05-18 01:33:03,460 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/history.txt.appattempt_1463535183398_0001_000000, maxErrors=10
2016-05-18 01:33:03,461 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:03,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535183398_0001_000000, appSubmitTime=1463535183445, launchTime=1463535183445
2016-05-18 01:33:03,465 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:03,469 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53363
2016-05-18 01:33:03,470 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535183398
2016-05-18 01:33:03,471 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535183398_0001_000000, startTime=1463535183471
2016-05-18 01:33:03,472 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:03,529 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:03,530 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:03,530 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001, dagName=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,549 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,549 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:03,550 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535183398_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/dag_1463535183398_0001_1.dot
2016-05-18 01:33:03,551 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/dag_1463535183398_0001_1-tez-dag.pb.txt
2016-05-18 01:33:03,567 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183398_0001_1
2016-05-18 01:33:03,568 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,568 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03 Running Dag: dag_1463535183398_0001_1
2016-05-18 01:33:03,583 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535183398_0001_1, submitTime=1463535183549
2016-05-18 01:33:03,585 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001/tez-conf.pb" } size: 40938 timestamp: 1463535183000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:03,585 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,585 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,585 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:03,585 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:03,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535183398_0001_1, initTime=1463535183585
2016-05-18 01:33:03,587 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:03,587 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:03,588 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_STARTED]: dagID=dag_1463535183398_0001_1, startTime=1463535183588
2016-05-18 01:33:03,588 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:03,588 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D89E8741659F4A4AB5D517E5101AB46F : {823A31594A604F9196990374D71566E0={InputName=823A31594A604F9196990374D71566E0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:03,588 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 823A31594A604F9196990374D71566E0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:03,588 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001, dagName=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,590 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:03,590 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 823A31594A604F9196990374D71566E0 on vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,592 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 823A31594A604F9196990374D71566E0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:03,595 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:03,595 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:33:03,596 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:03,596 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:03,596 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 823A31594A604F9196990374D71566E0 on vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,598 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,598 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {823A31594A604F9196990374D71566E0=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:03,599 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] parallelism set to 1
2016-05-18 01:33:03,599 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,599 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], numAdditionalOutputs=1
2016-05-18 01:33:03,600 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=555D641D5040473BA1A557F5187EB3DD, vertex=vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:03,602 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D89E8741659F4A4AB5D517E5101AB46F:555D641D5040473BA1A557F5187EB3DD using oldmapred API
2016-05-18 01:33:03,602 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=555D641D5040473BA1A557F5187EB3DD, vertexName=D89E8741659F4A4AB5D517E5101AB46F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:03,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, vertexId=vertex_1463535183398_0001_1_00, initRequestedTime=1463535183588, initedTime=1463535183599, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:03,603 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] with distanceFromRoot: 0
2016-05-18 01:33:03,604 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:03,604 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535183398_0001_1_00, startRequestedTime=1463535183590, startedTime=1463535183604
2016-05-18 01:33:03,604 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535183398_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:03,604 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:03,604 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D89E8741659F4A4AB5D517E5101AB46F
2016-05-18 01:33:03,605 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,609 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_STARTED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskId=task_1463535183398_0001_1_00_000000, scheduledTime=1463535183605, launchTime=1463535183605
2016-05-18 01:33:03,609 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,610 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:03,611 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535183398_0001, with dag id: dag_1463535183398_0001_1
2016-05-18 01:33:03,613 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535183398_0001_00_000001, launchTime=1463535183613
2016-05-18 01:33:03,614 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,615 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535183398_0001_1_00_000000_0] started. Is using containerId: [container_1463535183398_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:03,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, startTime=1463535183615, containerId=container_1463535183398_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:03,618 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183398_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,619 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,619 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,619 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:03,620 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, VertexName: D89E8741659F4A4AB5D517E5101AB46F, VertexParallelism: 1, TaskAttemptID:attempt_1463535183398_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=823A31594A604F9196990374D71566E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=555D641D5040473BA1A557F5187EB3DD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:03,620 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:03,621 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,621 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535183398_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:03,622 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535183398_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:03,628 INFO  input.MRInput (MRInput.java:initialize(440)) - 823A31594A604F9196990374D71566E0 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:03,628 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,630 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:03,641 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, ordinal: 0
2016-05-18 01:33:03,641 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, mem on start (mb), free: 351, total: 395, max: 672
2016-05-18 01:33:03,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:03,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:03,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:03,642 INFO  output.MROutput (MROutput.java:initialize(421)) - 555D641D5040473BA1A557F5187EB3DD: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:03,643 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:03,644 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[823A31594A604F9196990374D71566E0:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [555D641D5040473BA1A557F5187EB3DD:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:03,644 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:03,645 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:03,646 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:03,646 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[823A31594A604F9196990374D71566E0:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [555D641D5040473BA1A557F5187EB3DD:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:03,646 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:03,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 823A31594A604F9196990374D71566E0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:03,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:03,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:03,647 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:03,648 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,650 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:33:03,650 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 823A31594A604F9196990374D71566E0 initialized RecordReader from event
2016-05-18 01:33:03,651 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,652 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,652 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"], id: 555D641D5040473BA1A557F5187EB3DD
2016-05-18 01:33:03,653 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"]
2016-05-18 01:33:03,653 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,653 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:03,656 INFO  output.MROutput (MROutput.java:close(526)) - 555D641D5040473BA1A557F5187EB3DD closed
2016-05-18 01:33:03,656 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535183398_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:03,657 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,658 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, mem on close (mb), free: 349, total: 395, max: 672
2016-05-18 01:33:03,658 INFO  output.MROutput (MROutput.java:close(526)) - 555D641D5040473BA1A557F5187EB3DD closed
2016-05-18 01:33:03,658 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,659 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:03,659 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535183398_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:03,660 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535183398_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1][cascading.flow.SliceCounters Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=1]]
2016-05-18 01:33:03,661 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535183398_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:03,661 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,661 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:03,661 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,661 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,662 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, creationTime=1463535183609, allocationTime=1463535183612, startTime=1463535183615, finishTime=1463535183661, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:03,662 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,662 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskId=task_1463535183398_0001_1_00_000000, startTime=1463535183615, finishTime=1463535183662, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463535183398_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1
2016-05-18 01:33:03,663 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:03,664 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,664 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, vertexId=vertex_1463535183398_0001_1_00, initRequestedTime=1463535183588, initedTime=1463535183599, startRequestedTime=1463535183590, startedTime=1463535183604, finishTime=1463535183664, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1, vertexStats=firstTaskStartTime=1463535183615, firstTasksToStart=[ task_1463535183398_0001_1_00_000000 ], lastTaskFinishTime=1463535183661, lastTasksToFinish=[ task_1463535183398_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535183398_0001_1_00_000000 ], longestDurationTasks=[ task_1463535183398_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:03,665 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:03,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535183398_0001_00_000001, stoppedTime=1463535183665, exitStatus=0
2016-05-18 01:33:03,665 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:03,665 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,665 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535183398_0001_1
2016-05-18 01:33:03,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535183398_0001_1
2016-05-18 01:33:03,665 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:03,665 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 555D641D5040473BA1A557F5187EB3DD for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], outputName: 555D641D5040473BA1A557F5187EB3DD
2016-05-18 01:33:03,666 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535183398_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:03,672 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535183398_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:03,672 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183398_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:03,673 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535183398_0001_00_000001, Exiting
2016-05-18 01:33:03,673 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,673 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,673 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535183398_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:03,673 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535183398_0001_00_000001 completed successfully
2016-05-18 01:33:03,677 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:555D641D5040473BA1A557F5187EB3DD of vertex/vertexGroup:D89E8741659F4A4AB5D517E5101AB46F isVertexGroupOutput:false
2016-05-18 01:33:03,677 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,678 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535183398_0001_1, queueSize=0
2016-05-18 01:33:03,679 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535183398_0001_1, startTime=1463535183587, finishTime=1463535183677, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1
2016-05-18 01:33:03,679 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535183398_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:03,679 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:03 Completed Dag: dag_1463535183398_0001_1
2016-05-18 01:33:03,679 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535183398_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:03,679 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183398_0001_1_post
2016-05-18 01:33:03,679 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,679 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:03,680 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:03,680 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, with id=dag_1463535183398_0001_1
2016-05-18 01:33:03,680 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, with id=dag_1463535183398_0001_1
2016-05-18 01:33:03,687 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001
2016-05-18 01:33:03,689 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:03,689 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:03,689 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:03,690 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:03,690 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:03,690 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:03,691 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:03,691 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:03,691 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:03,692 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,692 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:03,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:03,693 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,694 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:03,697 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001
2016-05-18 01:33:03,698 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand/_temporary
2016-05-18 01:33:03,699 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.332
2016-05-18 01:33:03,737 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand
2016-05-18 01:33:03,767 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:03,768 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:03,814 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 01:33:03,826 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 01:33:03,826 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,827 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,827 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:03,845 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:33:03,846 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor
2016-05-18 01:33:03,861 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:03,862 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,862 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"]
2016-05-18 01:33:03,863 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:03,863 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:03,864 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:03,866 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...egression/complexlogicxor
2016-05-18 01:33:03,866 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:03,882 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:03,886 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,886 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:03,886 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:03,895 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001 doesn't exist and is created
2016-05-18 01:33:03,918 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:03,918 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:03,919 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd
2016-05-18 01:33:03,935 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535183886_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,936 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,936 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,936 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:03,936 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:03,939 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:03,941 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:03,942 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:03,943 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:03,943 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:03,943 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:03,944 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:03,944 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:03,944 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:03,944 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:03,944 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,944 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,945 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:03,946 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:03,948 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:03,949 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:03,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:03,949 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir
2016-05-18 01:33:03,950 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/history.txt.appattempt_1463535183886_0001_000000, maxErrors=10
2016-05-18 01:33:03,950 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:03,951 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535183886_0001_000000, appSubmitTime=1463535183935, launchTime=1463535183935
2016-05-18 01:33:03,957 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:03,960 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51435
2016-05-18 01:33:03,962 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535183886
2016-05-18 01:33:03,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535183886_0001_000000, startTime=1463535183964
2016-05-18 01:33:03,964 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:04,019 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:04,019 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:04,019 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001, dagName=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,046 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,046 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:04,048 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535183886_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/dag_1463535183886_0001_1.dot
2016-05-18 01:33:04,048 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/dag_1463535183886_0001_1-tez-dag.pb.txt
2016-05-18 01:33:04,053 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183886_0001_1
2016-05-18 01:33:04,054 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,054 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04 Running Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04,070 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535183886_0001_1, submitTime=1463535184046
2016-05-18 01:33:04,071 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001/tez-conf.pb" } size: 40938 timestamp: 1463535183000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,071 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,071 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,071 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,071 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:04,072 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535183886_0001_1, initTime=1463535184072
2016-05-18 01:33:04,073 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:04,074 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:04,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_STARTED]: dagID=dag_1463535183886_0001_1, startTime=1463535184074
2016-05-18 01:33:04,074 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:04,074 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F57C5A953E9E49E2AB619F031E636965 : {8C7E769C0A914B11A4A6BF1CB33FAB0C={InputName=8C7E769C0A914B11A4A6BF1CB33FAB0C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,074 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 8C7E769C0A914B11A4A6BF1CB33FAB0C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,074 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,076 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,076 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C on vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,076 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001, dagName=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,078 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 8C7E769C0A914B11A4A6BF1CB33FAB0C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,081 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,081 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:33:04,081 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,081 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:04,082 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C on vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,083 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,083 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {8C7E769C0A914B11A4A6BF1CB33FAB0C=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,086 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] parallelism set to 1
2016-05-18 01:33:04,086 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,086 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], numAdditionalOutputs=1
2016-05-18 01:33:04,086 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8C29811579D948F788C751B2257506DF, vertex=vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:04,089 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F57C5A953E9E49E2AB619F031E636965:8C29811579D948F788C751B2257506DF using oldmapred API
2016-05-18 01:33:04,089 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8C29811579D948F788C751B2257506DF, vertexName=F57C5A953E9E49E2AB619F031E636965, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:04,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F57C5A953E9E49E2AB619F031E636965, vertexId=vertex_1463535183886_0001_1_00, initRequestedTime=1463535184074, initedTime=1463535184086, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,090 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] with distanceFromRoot: 0
2016-05-18 01:33:04,090 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535183886_0001_1_00, startRequestedTime=1463535184076, startedTime=1463535184091
2016-05-18 01:33:04,092 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535183886_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,092 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,092 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F57C5A953E9E49E2AB619F031E636965
2016-05-18 01:33:04,092 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_STARTED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskId=task_1463535183886_0001_1_00_000000, scheduledTime=1463535184093, launchTime=1463535184093
2016-05-18 01:33:04,094 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,095 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:04,097 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535183886_0001, with dag id: dag_1463535183886_0001_1
2016-05-18 01:33:04,098 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535183886_0001_00_000001, launchTime=1463535184097
2016-05-18 01:33:04,099 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,100 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535183886_0001_1_00_000000_0] started. Is using containerId: [container_1463535183886_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,100 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, startTime=1463535184099, containerId=container_1463535183886_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:04,102 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183886_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,102 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,103 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,103 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,103 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, VertexName: F57C5A953E9E49E2AB619F031E636965, VertexParallelism: 1, TaskAttemptID:attempt_1463535183886_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C7E769C0A914B11A4A6BF1CB33FAB0C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8C29811579D948F788C751B2257506DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:04,103 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,104 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,104 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535183886_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:04,105 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535183886_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,108 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,112 INFO  input.MRInput (MRInput.java:initialize(440)) - 8C7E769C0A914B11A4A6BF1CB33FAB0C using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,112 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,116 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F57C5A953E9E49E2AB619F031E636965, ordinal: 0
2016-05-18 01:33:04,116 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57C5A953E9E49E2AB619F031E636965, mem on start (mb), free: 271, total: 395, max: 672
2016-05-18 01:33:04,116 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,116 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,117 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,122 INFO  output.MROutput (MROutput.java:initialize(421)) - 8C29811579D948F788C751B2257506DF: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:04,122 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,123 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8C29811579D948F788C751B2257506DF:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [8C7E769C0A914B11A4A6BF1CB33FAB0C:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 01:33:04,123 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,123 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,123 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:04,123 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8C29811579D948F788C751B2257506DF:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [8C7E769C0A914B11A4A6BF1CB33FAB0C:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 01:33:04,123 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,124 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,125 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,125 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,126 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:33:04,126 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 8C7E769C0A914B11A4A6BF1CB33FAB0C initialized RecordReader from event
2016-05-18 01:33:04,129 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:04,130 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,130 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"], id: 8C29811579D948F788C751B2257506DF
2016-05-18 01:33:04,131 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"]
2016-05-18 01:33:04,131 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:04,131 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F57C5A953E9E49E2AB619F031E636965, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,134 INFO  output.MROutput (MROutput.java:close(526)) - 8C29811579D948F788C751B2257506DF closed
2016-05-18 01:33:04,135 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535183886_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:04,136 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,136 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57C5A953E9E49E2AB619F031E636965, mem on close (mb), free: 269, total: 395, max: 672
2016-05-18 01:33:04,137 INFO  output.MROutput (MROutput.java:close(526)) - 8C29811579D948F788C751B2257506DF closed
2016-05-18 01:33:04,137 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,138 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:04,138 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535183886_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:04,139 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535183886_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=3]]
2016-05-18 01:33:04,140 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535183886_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:04,140 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,140 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:04,140 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,141 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, creationTime=1463535184094, allocationTime=1463535184096, startTime=1463535184099, finishTime=1463535184140, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,141 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,143 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskId=task_1463535183886_0001_1_00_000000, startTime=1463535184099, finishTime=1463535184143, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463535183886_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3
2016-05-18 01:33:04,144 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,144 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,142 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, vertexId=vertex_1463535183886_0001_1_00, initRequestedTime=1463535184074, initedTime=1463535184086, startRequestedTime=1463535184076, startedTime=1463535184091, finishTime=1463535184144, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3, vertexStats=firstTaskStartTime=1463535184099, firstTasksToStart=[ task_1463535183886_0001_1_00_000000 ], lastTaskFinishTime=1463535184140, lastTasksToFinish=[ task_1463535183886_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535183886_0001_1_00_000000 ], longestDurationTasks=[ task_1463535183886_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,145 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535183886_0001_00_000001, stoppedTime=1463535184145, exitStatus=0
2016-05-18 01:33:04,146 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:04,146 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,146 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535183886_0001_1
2016-05-18 01:33:04,146 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535183886_0001_1
2016-05-18 01:33:04,146 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:04,146 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8C29811579D948F788C751B2257506DF for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], outputName: 8C29811579D948F788C751B2257506DF
2016-05-18 01:33:04,147 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535183886_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,155 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535183886_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,155 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183886_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,155 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535183886_0001_00_000001, Exiting
2016-05-18 01:33:04,155 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,155 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,156 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535183886_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,156 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8C29811579D948F788C751B2257506DF of vertex/vertexGroup:F57C5A953E9E49E2AB619F031E636965 isVertexGroupOutput:false
2016-05-18 01:33:04,156 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,156 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535183886_0001_00_000001 completed successfully
2016-05-18 01:33:04,158 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535183886_0001_1, queueSize=0
2016-05-18 01:33:04,158 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535183886_0001_1, startTime=1463535184074, finishTime=1463535184157, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3
2016-05-18 01:33:04,159 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535183886_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:04,159 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:04 Completed Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04,159 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535183886_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:04,159 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183886_0001_1_post
2016-05-18 01:33:04,160 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,160 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:04,160 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:04,160 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, with id=dag_1463535183886_0001_1
2016-05-18 01:33:04,161 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, with id=dag_1463535183886_0001_1
2016-05-18 01:33:04,166 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001
2016-05-18 01:33:04,169 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:04,169 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:04,169 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:04,169 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:04,169 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:04,169 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:04,170 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:04,170 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:04,170 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:04,170 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,171 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:04,171 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:04,171 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,171 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:04,175 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001
2016-05-18 01:33:04,176 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor/_temporary
2016-05-18 01:33:04,176 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.314
2016-05-18 01:33:04,214 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor
2016-05-18 01:33:04,244 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:04,245 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:04,302 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 01:33:04,314 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-05-18 01:33:04,315 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:04,315 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:04,315 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:04,343 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:33:04,356 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt
2016-05-18 01:33:04,369 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost
2016-05-18 01:33:04,488 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-18 01:33:04,489 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,489 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"]
2016-05-18 01:33:04,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  parallel execution of steps is enabled: false
2016-05-18 01:33:04,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  executing total steps: 1
2016-05-18 01:33:04,490 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  allocating management threads: 1
2016-05-18 01:33:04,492 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,492 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:04,507 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-18 01:33:04,511 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:04,511 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:04,511 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:04,521 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001 doesn't exist and is created
2016-05-18 01:33:04,541 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:04,541 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:04,542 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd
2016-05-18 01:33:04,557 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535184511_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:04,557 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:04,557 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:04,558 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:04,558 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:04,558 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:04,559 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:04,559 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:04,561 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:04,561 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:04,562 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:04,562 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:04,563 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:04,564 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:04,564 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:04,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir
2016-05-18 01:33:04,565 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/history.txt.appattempt_1463535184511_0001_000000, maxErrors=10
2016-05-18 01:33:04,565 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:04,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535184511_0001_000000, appSubmitTime=1463535184556, launchTime=1463535184556
2016-05-18 01:33:04,574 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:04,576 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56303
2016-05-18 01:33:04,577 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535184511
2016-05-18 01:33:04,579 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535184511_0001_000000, startTime=1463535184579
2016-05-18 01:33:04,579 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:04,641 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:04,641 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:04,642 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001, dagName=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,662 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,663 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:04,664 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535184511_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/dag_1463535184511_0001_1.dot
2016-05-18 01:33:04,665 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/dag_1463535184511_0001_1-tez-dag.pb.txt
2016-05-18 01:33:04,676 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535184511_0001_1
2016-05-18 01:33:04,676 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,676 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04 Running Dag: dag_1463535184511_0001_1
2016-05-18 01:33:04,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535184511_0001_1, submitTime=1463535184663
2016-05-18 01:33:04,695 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,695 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,696 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,697 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,697 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,697 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,698 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,698 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,699 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,699 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:04,700 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535184511_0001_1, initTime=1463535184699
2016-05-18 01:33:04,700 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:04,701 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:04,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_STARTED]: dagID=dag_1463535184511_0001_1, startTime=1463535184701
2016-05-18 01:33:04,701 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:04,701 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 : {DCB1B28207D542E481A43879144C97C8={InputName=DCB1B28207D542E481A43879144C97C8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,701 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DCB1B28207D542E481A43879144C97C8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,701 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,701 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,701 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,702 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,703 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,703 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E0B459EF7267495B95C50BAEE54197F0 : {79BC5C67842049FA937D12E1717784E6={InputName=79BC5C67842049FA937D12E1717784E6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,703 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 79BC5C67842049FA937D12E1717784E6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,703 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DCB1B28207D542E481A43879144C97C8 on vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,703 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,704 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001, dagName=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,704 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,704 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,705 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,706 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,706 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,710 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 79BC5C67842049FA937D12E1717784E6 on vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,711 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:04,711 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,712 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:04,714 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], numAdditionalOutputs=1
2016-05-18 01:33:04,714 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=BBA6B75524E64A87A4E580C0A2F9EF1E, vertex=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:04,714 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DCB1B28207D542E481A43879144C97C8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,719 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D952A5E7D9C44D339A00FECCD6FCD420:BBA6B75524E64A87A4E580C0A2F9EF1E using oldmapred API
2016-05-18 01:33:04,721 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=BBA6B75524E64A87A4E580C0A2F9EF1E, vertexName=D952A5E7D9C44D339A00FECCD6FCD420, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:04,722 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 79BC5C67842049FA937D12E1717784E6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,722 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, initRequestedTime=1463535184706, initedTime=1463535184714, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:04,723 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:04,723 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,723 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:04,723 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,724 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:04,724 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,724 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:04,724 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,725 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:04,725 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 79BC5C67842049FA937D12E1717784E6 on vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,724 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DCB1B28207D542E481A43879144C97C8 on vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,728 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,728 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {79BC5C67842049FA937D12E1717784E6=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,728 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] parallelism set to 1
2016-05-18 01:33:04,728 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, vertexId=vertex_1463535184511_0001_1_00, initRequestedTime=1463535184703, initedTime=1463535184729, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,729 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] with distanceFromRoot: 0
2016-05-18 01:33:04,729 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,731 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_00, startRequestedTime=1463535184706, startedTime=1463535184731
2016-05-18 01:33:04,731 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,731 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,731 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535184511_0001_1_00 for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 1 numSources: 2
2016-05-18 01:33:04,732 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,732 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,732 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DCB1B28207D542E481A43879144C97C8=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,732 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] parallelism set to 1
2016-05-18 01:33:04,733 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,732 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,732 INFO  impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 1 numSources: 2
2016-05-18 01:33:04,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, vertexId=vertex_1463535184511_0001_1_01, initRequestedTime=1463535184701, initedTime=1463535184733, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,734 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] with distanceFromRoot: 0
2016-05-18 01:33:04,734 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,735 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskId=task_1463535184511_0001_1_00_000000, scheduledTime=1463535184734, launchTime=1463535184734
2016-05-18 01:33:04,736 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_01, startRequestedTime=1463535184706, startedTime=1463535184735
2016-05-18 01:33:04,736 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,736 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,737 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,737 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,737 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,737 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535184511_0001_1_01 for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 2 numSources: 2
2016-05-18 01:33:04,738 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] with distanceFromRoot: 1
2016-05-18 01:33:04,738 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskId=task_1463535184511_0001_1_01_000000, scheduledTime=1463535184738, launchTime=1463535184738
2016-05-18 01:33:04,739 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_02, startRequestedTime=1463535184737, startedTime=1463535184738
2016-05-18 01:33:04,739 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,740 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D952A5E7D9C44D339A00FECCD6FCD420 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:04,741 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E0B459EF7267495B95C50BAEE54197F0 in vertex: D952A5E7D9C44D339A00FECCD6FCD420 numBipartiteSourceTasks: 1
2016-05-18 01:33:04,741 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 in vertex: D952A5E7D9C44D339A00FECCD6FCD420 numBipartiteSourceTasks: 2
2016-05-18 01:33:04,741 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,754 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:04,757 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000001, launchTime=1463535184756
2016-05-18 01:33:04,757 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000002, launchTime=1463535184757
2016-05-18 01:33:04,757 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,759 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_00_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,759 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,759 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, startTime=1463535184758, containerId=container_1463535184511_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:04,760 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,761 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,761 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,762 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,761 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_01_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,762 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,762 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, startTime=1463535184761, containerId=container_1463535184511_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:04,762 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: E0B459EF7267495B95C50BAEE54197F0, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79BC5C67842049FA937D12E1717784E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:04,762 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,766 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463535184511_0001, with dag id: dag_1463535184511_0001_1
2016-05-18 01:33:04,767 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,767 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,767 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,768 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,768 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: FF2A0F3C18AB4F6EBCF105A139B26BC5, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DCB1B28207D542E481A43879144C97C8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:04,768 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,769 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535184511_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:04,769 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,773 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,774 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,781 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535184511_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:33:04,781 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,785 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,789 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, ordinal: 1
2016-05-18 01:33:04,789 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, mem on start (mb), free: 353, total: 395, max: 672
2016-05-18 01:33:04,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,789 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,791 INFO  input.MRInput (MRInput.java:initialize(440)) - 79BC5C67842049FA937D12E1717784E6 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,792 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,792 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,793 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,793 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[79BC5C67842049FA937D12E1717784E6:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D952A5E7D9C44D339A00FECCD6FCD420:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:04,794 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,798 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:04,798 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[79BC5C67842049FA937D12E1717784E6:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D952A5E7D9C44D339A00FECCD6FCD420:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:04,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,799 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 79BC5C67842049FA937D12E1717784E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,800 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,801 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,798 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, ordinal: 0
2016-05-18 01:33:04,802 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, mem on start (mb), free: 352, total: 395, max: 672
2016-05-18 01:33:04,802 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,803 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,801 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,798 INFO  input.MRInput (MRInput.java:initialize(440)) - DCB1B28207D542E481A43879144C97C8 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,803 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:04,804 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,804 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 79BC5C67842049FA937D12E1717784E6 initialized RecordReader from event
2016-05-18 01:33:04,805 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,805 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DCB1B28207D542E481A43879144C97C8:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D952A5E7D9C44D339A00FECCD6FCD420:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:04,805 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,806 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,806 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:04,806 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DCB1B28207D542E481A43879144C97C8:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D952A5E7D9C44D339A00FECCD6FCD420:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:04,807 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,807 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,807 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DCB1B28207D542E481A43879144C97C8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,808 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,808 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,808 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,809 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,811 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,811 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"] streamed: true, id: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,811 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']], id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,812 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,813 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,814 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:04,814 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - DCB1B28207D542E481A43879144C97C8 initialized RecordReader from event
2016-05-18 01:33:04,815 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D952A5E7D9C44D339A00FECCD6FCD420 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@21fc2741, comparator=cascading.tuple.hadoop.util.TupleComparator@5ba78e1c, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:04,814 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,815 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']], id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,816 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,816 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false
2016-05-18 01:33:04,819 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D952A5E7D9C44D339A00FECCD6FCD420 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5562ff46, comparator=cascading.tuple.hadoop.util.TupleComparator@56657875, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:04,820 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false
2016-05-18 01:33:04,832 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:04,832 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:04,833 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:04,833 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D952A5E7D9C44D339A00FECCD6FCD420: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:04,833 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,833 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,841 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:04,842 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:04,842 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:04,842 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D952A5E7D9C44D339A00FECCD6FCD420: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:04,842 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,843 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,844 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,844 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, mem on close (mb), free: 149, total: 395, max: 672
2016-05-18 01:33:04,844 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D952A5E7D9C44D339A00FECCD6FCD420: Starting flush of map output
2016-05-18 01:33:04,845 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D952A5E7D9C44D339A00FECCD6FCD420: Span0.length = 5, perItem = 15
2016-05-18 01:33:04,845 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D952A5E7D9C44D339A00FECCD6FCD420: done sorting span=0, length=5, time=0
2016-05-18 01:33:04,845 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D952A5E7D9C44D339A00FECCD6FCD420: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:04,847 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D952A5E7D9C44D339A00FECCD6FCD420: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_00_000000_0_10041_0/file.out
2016-05-18 01:33:04,847 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,848 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, mem on close (mb), free: 148, total: 395, max: 672
2016-05-18 01:33:04,848 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D952A5E7D9C44D339A00FECCD6FCD420: Starting flush of map output
2016-05-18 01:33:04,848 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D952A5E7D9C44D339A00FECCD6FCD420: Span0.length = 5, perItem = 15
2016-05-18 01:33:04,849 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D952A5E7D9C44D339A00FECCD6FCD420: done sorting span=0, length=5, time=0
2016-05-18 01:33:04,849 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D952A5E7D9C44D339A00FECCD6FCD420: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:04,850 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D952A5E7D9C44D339A00FECCD6FCD420: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_01_000000_0_10043_0/file.out
2016-05-18 01:33:04,854 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,854 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:04,855 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:04,855 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:04,856 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,857 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:04,857 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:04,858 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,858 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:04,858 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,857 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, creationTime=1463535184735, allocationTime=1463535184755, startTime=1463535184758, finishTime=1463535184857, timeTaken=99, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,858 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,858 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:04,859 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,859 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:04,860 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskId=task_1463535184511_0001_1_00_000000, startTime=1463535184758, finishTime=1463535184859, timeTaken=101, status=SUCCEEDED, successfulAttemptID=attempt_1463535184511_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:04,860 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,860 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,862 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:04,863 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, vertexId=vertex_1463535184511_0001_1_00, initRequestedTime=1463535184703, initedTime=1463535184729, startRequestedTime=1463535184706, startedTime=1463535184731, finishTime=1463535184860, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535184758, firstTasksToStart=[ task_1463535184511_0001_1_00_000000 ], lastTaskFinishTime=1463535184857, lastTasksToFinish=[ task_1463535184511_0001_1_00_000000 ], minTaskDuration=99, maxTaskDuration=99, avgTaskDuration=99.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535184511_0001_1_00_000000 ], longestDurationTasks=[ task_1463535184511_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,863 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,863 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,863 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:04,863 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,863 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,863 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] attempt: attempt_1463535184511_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:04,865 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000001, stoppedTime=1463535184865, exitStatus=0
2016-05-18 01:33:04,866 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, creationTime=1463535184738, allocationTime=1463535184755, startTime=1463535184761, finishTime=1463535184865, timeTaken=104, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,866 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,866 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,867 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535184511_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,867 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,868 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskId=task_1463535184511_0001_1_01_000000, startTime=1463535184761, finishTime=1463535184867, timeTaken=106, status=SUCCEEDED, successfulAttemptID=attempt_1463535184511_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:04,869 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,869 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,869 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535184511_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,870 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,870 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535184511_0001_00_000001, Exiting
2016-05-18 01:33:04,870 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,870 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,871 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,871 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535184511_0001_00_000001 completed successfully
2016-05-18 01:33:04,883 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, vertexId=vertex_1463535184511_0001_1_01, initRequestedTime=1463535184701, initedTime=1463535184733, startRequestedTime=1463535184706, startedTime=1463535184735, finishTime=1463535184869, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535184761, firstTasksToStart=[ task_1463535184511_0001_1_01_000000 ], lastTaskFinishTime=1463535184865, lastTasksToFinish=[ task_1463535184511_0001_1_01_000000 ], minTaskDuration=104, maxTaskDuration=104, avgTaskDuration=104.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535184511_0001_1_01_000000 ], longestDurationTasks=[ task_1463535184511_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,883 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,884 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000002, stoppedTime=1463535184884, exitStatus=0
2016-05-18 01:33:04,884 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] attempt: attempt_1463535184511_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:04,885 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D952A5E7D9C44D339A00FECCD6FCD420
2016-05-18 01:33:04,885 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:04,885 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E0B459EF7267495B95C50BAEE54197F0, destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:04,885 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,886 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,886 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,886 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535184511_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,887 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskId=task_1463535184511_0001_1_02_000000, scheduledTime=1463535184886, launchTime=1463535184886
2016-05-18 01:33:04,887 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,889 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000003, launchTime=1463535184889
2016-05-18 01:33:04,890 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,890 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_02_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,890 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0, startTime=1463535184890, containerId=container_1463535184511_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:04,891 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,892 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,892 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,892 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,892 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: D952A5E7D9C44D339A00FECCD6FCD420, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=E0B459EF7267495B95C50BAEE54197F0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BBA6B75524E64A87A4E580C0A2F9EF1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:04,892 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,893 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,895 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:04,896 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:04,897 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535184511_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,898 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:04,900 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,903 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535184511_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,903 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000002: 39 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,904 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535184511_0001_00_000002, Exiting
2016-05-18 01:33:04,904 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,904 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,904 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,904 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535184511_0001_00_000002 completed successfully
2016-05-18 01:33:04,909 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, ordinal: 2
2016-05-18 01:33:04,909 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, mem on start (mb), free: 358, total: 395, max: 672
2016-05-18 01:33:04,909 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,909 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:04,909 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,909 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,913 INFO  output.MROutput (MROutput.java:initialize(421)) - BBA6B75524E64A87A4E580C0A2F9EF1E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:04,913 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,913 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E0B459EF7267495B95C50BAEE54197F0:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [FF2A0F3C18AB4F6EBCF105A139B26BC5:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [BBA6B75524E64A87A4E580C0A2F9EF1E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:04,913 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,914 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:04,914 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:04,914 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E0B459EF7267495B95C50BAEE54197F0:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [FF2A0F3C18AB4F6EBCF105A139B26BC5:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [BBA6B75524E64A87A4E580C0A2F9EF1E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:04,914 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,914 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FF2A0F3C18AB4F6EBCF105A139B26BC5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,915 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:04,915 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:04,916 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:04,917 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E0B459EF7267495B95C50BAEE54197F0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,917 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,917 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:33:04,917 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:33:04,918 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,918 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E0B459EF7267495B95C50BAEE54197F0: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:04,918 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E0B459EF7267495B95C50BAEE54197F0: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:04,919 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E0B459EF7267495B95C50BAEE54197F0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:04,920 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,921 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,921 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,925 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E0B459EF7267495B95C50BAEE54197F0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:04,927 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:04,927 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']]
2016-05-18 01:33:04,927 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] streamed: true, id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,928 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"], id: BBA6B75524E64A87A4E580C0A2F9EF1E
2016-05-18 01:33:04,928 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"]
2016-05-18 01:33:04,928 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96, for 2 inputs
2016-05-18 01:33:04,928 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96, for 2 inputs
2016-05-18 01:33:04,929 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:04,930 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:04,930 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535184511_0001_1_00_000000_0_10041} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535184930, TimeTaken=3, Rate=0.03 MB/s
2016-05-18 01:33:04,930 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,931 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,931 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535184511_0001_1_01_000000_0_10043} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535184931, TimeTaken=4, Rate=0.02 MB/s
2016-05-18 01:33:04,932 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,932 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E0B459EF7267495B95C50BAEE54197F0, wasInterrupted=false
2016-05-18 01:33:04,932 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,932 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,932 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FF2A0F3C18AB4F6EBCF105A139B26BC5, wasInterrupted=false
2016-05-18 01:33:04,933 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_00_000000_0_10041/file.out, len=95, isLocal=true
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_01_000000_0_10043/file.out, len=95, isLocal=true
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:04,935 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:04,935 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:04,935 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,935 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: Shuffle Runner thread complete
2016-05-18 01:33:04,934 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:04,935 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:04,936 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:04,936 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,936 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E0B459EF7267495B95C50BAEE54197F0: Shuffle Runner thread complete
2016-05-18 01:33:04,936 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, all 2 inputs ready in: 00:00:00.006
2016-05-18 01:33:04,937 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:04,939 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E0B459EF7267495B95C50BAEE54197F0: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:04,942 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught OutOfMemoryException, will not trap, rethrowing
java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
	at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:04,946 INFO  output.MROutput (MROutput.java:close(526)) - BBA6B75524E64A87A4E580C0A2F9EF1E closed
2016-05-18 01:33:04,947 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535184511_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:04,948 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:04,949 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_02_000000_0: Counters: 33 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=8, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535184936, Process_Duration=12, Process_End_Time=1463535184948, Read_Duration=1, Tuples_Read=2]]
2016-05-18 01:33:04,949 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 01:33:04,950 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, mem on close (mb), free: 353, total: 395, max: 672
2016-05-18 01:33:04,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=D952A5E7D9C44D339A00FECCD6FCD420, index=2, interruptedStatus=false
2016-05-18 01:33:04,950 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,951 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:04,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=FF2A0F3C18AB4F6EBCF105A139B26BC5, interruptedStatus=true
2016-05-18 01:33:04,951 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,951 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E0B459EF7267495B95C50BAEE54197F0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:04,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=E0B459EF7267495B95C50BAEE54197F0, interruptedStatus=true
2016-05-18 01:33:04,952 INFO  output.MROutput (MROutput.java:close(526)) - BBA6B75524E64A87A4E580C0A2F9EF1E closed
2016-05-18 01:33:04,952 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=BBA6B75524E64A87A4E580C0A2F9EF1E, interruptedStatus=true
2016-05-18 01:33:04,954 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_02_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}  
2016-05-18 01:33:04,954 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:04,954 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,954 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,954 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000003 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=java.lang.OutOfMemoryError: fake error, errorMessage='TaskExecutionFailure: fake error'}
2016-05-18 01:33:04,954 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463535184511_0001_00_000003 completed but with errors
2016-05-18 01:33:04,954 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0, creationTime=1463535184887, allocationTime=1463535184888, startTime=1463535184890, finishTime=1463535184954, timeTaken=64, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
	at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 33, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535184936, Process_Duration=12, Process_End_Time=1463535184948, Read_Duration=1, Tuples_Read=2
2016-05-18 01:33:04,955 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463535184511_0001_00_000003 exited with diagnostics set to TaskExecutionFailure: fake error
2016-05-18 01:33:04,957 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000003, stoppedTime=1463535184957, exitStatus=1
2016-05-18 01:33:04,958 INFO  impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463535184511_0001_1_02_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 01:33:04,958 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskId=task_1463535184511_0001_1_02_000000, startTime=1463535184890, finishTime=1463535184958, timeTaken=68, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
	at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-18 01:33:04,958 INFO  impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] because task failed: task_1463535184511_0001_1_02_000000
2016-05-18 01:33:04,959 INFO  impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] due to trigger: OWN_TASK_FAILURE
2016-05-18 01:33:04,959 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 01:33:04,959 INFO  impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 01:33:04,959 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,959 INFO  impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=BBA6B75524E64A87A4E580C0A2F9EF1E, vertexId=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, initRequestedTime=1463535184706, initedTime=1463535184714, startRequestedTime=1463535184737, startedTime=1463535184738, finishTime=1463535184960, timeTaken=222, status=FAILED, diagnostics=Task failed, taskId=task_1463535184511_0001_1_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
	at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463535184890, firstTasksToStart=[ task_1463535184511_0001_1_02_000000 ], lastTaskFinishTime=1463535184954, lastTasksToFinish=[ task_1463535184511_0001_1_02_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[  ], longestDurationTasks=[  ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 01:33:04,961 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,962 INFO  node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463535184511_0001_1_02_000000_0 failed: true container: container_1463535184511_0001_00_000003 numFailedTAs: 0
2016-05-18 01:33:04,962 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] completed., numCompletedVertices=3, numSuccessfulVertices=2, numFailedVertices=1, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,962 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=2, numFailedVertices=1, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 01:33:04,962 INFO  impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 01:33:04,965 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535184511_0001_1, queueSize=0
2016-05-18 01:33:04,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535184511_0001_1, startTime=1463535184701, finishTime=1463535184965, timeTaken=264, status=FAILED, diagnostics=Vertex failed, vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, diagnostics=[Task failed, taskId=task_1463535184511_0001_1_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
	at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
	at cascading.TestFunction.operate(TestFunction.java:75)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
	at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
	at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
	at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
	at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
	at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
	at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=600, AM_GC_TIME_MILLIS=22, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, cascading.flow.SliceCounters, Process_Begin_Time=2927070369677, Process_Duration=14, Process_End_Time=2927070369691, Read_Duration=13, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:33:04,966 INFO  impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,966 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535184511_0001_1 finished with state: FAILED
2016-05-18 01:33:04,966 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:04 Completed Dag: dag_1463535184511_0001_1
2016-05-18 01:33:04,967 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535184511_0001_1, dagState=FAILED
2016-05-18 01:33:04,968 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535184511_0001_1_post
2016-05-18 01:33:04,968 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,968 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:04,968 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:04,969 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, with id=dag_1463535184511_0001_1
2016-05-18 01:33:04,969 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, with id=dag_1463535184511_0001_1
2016-05-18 01:33:04,972 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001
2016-05-18 01:33:04,976 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:04,976 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:04,976 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:04,977 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:04,977 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:04,977 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:04,977 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:04,979 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:04,979 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:04,979 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,979 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:04,980 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:04,980 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,981 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:04,985 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001
2016-05-18 01:33:04,985 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopping all jobs
2016-05-18 01:33:04,985 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopping: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,986 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopped all jobs
2016-05-18 01:33:04,986 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost/_temporary
2016-05-18 01:33:04,986 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper]  completed in: 00:00.498
exception = cascading.flow.FlowException: local step failed: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,987 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost
2016-05-18 01:33:05,015 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:05,016 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:05,061 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:05,073 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 01:33:05,073 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:05,073 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:05,074 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:05,090 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:33:05,097 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified
2016-05-18 01:33:05,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:05,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"]
2016-05-18 01:33:05,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:05,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:05,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:05,151 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,152 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:05,165 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:05,170 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,170 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:05,170 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:05,178 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001 doesn't exist and is created
2016-05-18 01:33:05,198 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:05,198 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:05,199 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd
2016-05-18 01:33:05,215 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535185170_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,216 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,216 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:05,220 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:05,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:05,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:05,223 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:05,223 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:05,223 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:05,223 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:05,224 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:05,224 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:05,224 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,224 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,225 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:05,225 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:05,227 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:05,227 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:05,228 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:05,228 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir
2016-05-18 01:33:05,228 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/history.txt.appattempt_1463535185170_0001_000000, maxErrors=10
2016-05-18 01:33:05,228 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:05,229 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535185170_0001_000000, appSubmitTime=1463535185215, launchTime=1463535185215
2016-05-18 01:33:05,237 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:05,241 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49105
2016-05-18 01:33:05,241 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535185170
2016-05-18 01:33:05,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535185170_0001_000000, startTime=1463535185243
2016-05-18 01:33:05,243 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:05,299 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:05,299 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:05,299 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001, dagName=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,317 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,317 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:05,318 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535185170_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/dag_1463535185170_0001_1.dot
2016-05-18 01:33:05,318 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/dag_1463535185170_0001_1-tez-dag.pb.txt
2016-05-18 01:33:05,331 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185170_0001_1
2016-05-18 01:33:05,331 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,331 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05 Running Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05,346 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535185170_0001_1, submitTime=1463535185317
2016-05-18 01:33:05,352 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/tez-conf.pb" } size: 40938 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,352 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,353 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,355 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/tez-conf.pb" } size: 40938 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,355 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,356 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,356 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:05,357 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535185170_0001_1, initTime=1463535185356
2016-05-18 01:33:05,358 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:05,358 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:05,358 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_STARTED]: dagID=dag_1463535185170_0001_1, startTime=1463535185358
2016-05-18 01:33:05,358 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001, dagName=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,359 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:05,359 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A1C305A30A0A4645A7EB63A6FAD6C8DE : {C8BFAA7939F444D3BAED37FF9C2C5159={InputName=C8BFAA7939F444D3BAED37FF9C2C5159}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:05,359 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C8BFAA7939F444D3BAED37FF9C2C5159, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:05,359 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,359 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,359 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,360 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,360 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:05,360 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,363 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C8BFAA7939F444D3BAED37FF9C2C5159 on vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,364 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:05,364 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,365 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,365 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185170_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:05,366 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], numAdditionalOutputs=1
2016-05-18 01:33:05,366 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=B2509D7F08954CF79D5A0301A1C4F73E, vertex=vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:05,367 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C8BFAA7939F444D3BAED37FF9C2C5159 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:05,370 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D751A2136EF14677A14F40147006203E:B2509D7F08954CF79D5A0301A1C4F73E using oldmapred API
2016-05-18 01:33:05,370 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=B2509D7F08954CF79D5A0301A1C4F73E, vertexName=D751A2136EF14677A14F40147006203E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:05,371 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:05,371 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:33:05,371 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:05,372 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:05,372 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C8BFAA7939F444D3BAED37FF9C2C5159 on vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,371 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D751A2136EF14677A14F40147006203E, vertexId=vertex_1463535185170_0001_1_01, initRequestedTime=1463535185360, initedTime=1463535185366, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:05,374 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:05,375 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,375 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C8BFAA7939F444D3BAED37FF9C2C5159=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:05,375 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] parallelism set to 1
2016-05-18 01:33:05,375 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,376 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, vertexId=vertex_1463535185170_0001_1_00, initRequestedTime=1463535185359, initedTime=1463535185376, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:05,376 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] with distanceFromRoot: 0
2016-05-18 01:33:05,377 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:05,377 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185170_0001_1_00, startRequestedTime=1463535185360, startedTime=1463535185377
2016-05-18 01:33:05,378 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185170_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:05,378 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,379 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,379 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,379 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535185170_0001_1_00 for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] numStartedSources: 1 numSources: 1
2016-05-18 01:33:05,379 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] with distanceFromRoot: 1
2016-05-18 01:33:05,381 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535185170_0001, with dag id: dag_1463535185170_0001_1
2016-05-18 01:33:05,384 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185170_0001_1_01, startRequestedTime=1463535185378, startedTime=1463535185380
2016-05-18 01:33:05,384 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,385 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D751A2136EF14677A14F40147006203E with 0 source tasks and 1 pending tasks
2016-05-18 01:33:05,385 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_STARTED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskId=task_1463535185170_0001_1_00_000000, scheduledTime=1463535185385, launchTime=1463535185385
2016-05-18 01:33:05,385 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,386 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A1C305A30A0A4645A7EB63A6FAD6C8DE in vertex: D751A2136EF14677A14F40147006203E numBipartiteSourceTasks: 1
2016-05-18 01:33:05,388 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:05,391 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185170_0001_00_000001, launchTime=1463535185391
2016-05-18 01:33:05,391 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,393 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,394 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185170_0001_1_00_000000_0] started. Is using containerId: [container_1463535185170_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,394 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, startTime=1463535185394, containerId=container_1463535185170_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,395 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,395 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,395 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,395 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, VertexName: A1C305A30A0A4645A7EB63A6FAD6C8DE, VertexParallelism: 1, TaskAttemptID:attempt_1463535185170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8BFAA7939F444D3BAED37FF9C2C5159, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D751A2136EF14677A14F40147006203E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:05,396 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,397 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,397 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535185170_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:05,397 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185170_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:05,401 INFO  input.MRInput (MRInput.java:initialize(440)) - C8BFAA7939F444D3BAED37FF9C2C5159 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:05,401 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,402 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,409 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, ordinal: 0
2016-05-18 01:33:05,409 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, mem on start (mb), free: 264, total: 395, max: 672
2016-05-18 01:33:05,409 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,410 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C8BFAA7939F444D3BAED37FF9C2C5159:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D751A2136EF14677A14F40147006203E:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:05,411 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,411 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,411 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:05,411 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C8BFAA7939F444D3BAED37FF9C2C5159:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D751A2136EF14677A14F40147006203E:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:05,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C8BFAA7939F444D3BAED37FF9C2C5159 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,413 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,413 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,415 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:33:05,415 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - C8BFAA7939F444D3BAED37FF9C2C5159 initialized RecordReader from event
2016-05-18 01:33:05,416 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,416 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,417 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'insert']], id: C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,417 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'insert']] C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,419 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D751A2136EF14677A14F40147006203E using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@16e73b4f, comparator=cascading.tuple.hadoop.util.TupleComparator@5ca1a3f6, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:05,419 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D751A2136EF14677A14F40147006203E: , UsingHashComparator=false
2016-05-18 01:33:05,431 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:05,432 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:05,432 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D751A2136EF14677A14F40147006203E: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:05,432 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D751A2136EF14677A14F40147006203E: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:05,432 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,433 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,435 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,435 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, mem on close (mb), free: 162, total: 395, max: 672
2016-05-18 01:33:05,436 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D751A2136EF14677A14F40147006203E: Starting flush of map output
2016-05-18 01:33:05,436 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D751A2136EF14677A14F40147006203E: Span0.length = 10, perItem = 16
2016-05-18 01:33:05,436 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D751A2136EF14677A14F40147006203E: done sorting span=0, length=10, time=0
2016-05-18 01:33:05,437 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D751A2136EF14677A14F40147006203E: Heap = SpanIterator<0:9> (span=Span[16777216,160]),
2016-05-18 01:33:05,438 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D751A2136EF14677A14F40147006203E: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-local-dir/output/attempt_1463535185170_0001_1_00_000000_0_10050_0/file.out
2016-05-18 01:33:05,443 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,444 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:05,444 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185170_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:05,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185170_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:33:05,449 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185170_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:05,449 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,449 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, creationTime=1463535185385, allocationTime=1463535185389, startTime=1463535185394, finishTime=1463535185449, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,449 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:05,450 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,450 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,451 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,451 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskId=task_1463535185170_0001_1_00_000000, startTime=1463535185394, finishTime=1463535185451, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463535185170_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:33:05,452 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,452 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,453 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, vertexId=vertex_1463535185170_0001_1_00, initRequestedTime=1463535185359, initedTime=1463535185376, startRequestedTime=1463535185360, startedTime=1463535185377, finishTime=1463535185452, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463535185394, firstTasksToStart=[ task_1463535185170_0001_1_00_000000 ], lastTaskFinishTime=1463535185449, lastTasksToFinish=[ task_1463535185170_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185170_0001_1_00_000000 ], longestDurationTasks=[ task_1463535185170_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,454 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,454 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185170_0001_00_000001, stoppedTime=1463535185454, exitStatus=0
2016-05-18 01:33:05,454 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] attempt: attempt_1463535185170_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:05,454 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:05,454 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D751A2136EF14677A14F40147006203E
2016-05-18 01:33:05,455 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,455 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, destinationVertexName=D751A2136EF14677A14F40147006203E. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:05,455 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,455 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185170_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,455 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_STARTED]: vertexName=D751A2136EF14677A14F40147006203E, taskId=task_1463535185170_0001_1_01_000000, scheduledTime=1463535185455, launchTime=1463535185455
2016-05-18 01:33:05,456 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,457 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185170_0001_00_000002, launchTime=1463535185457
2016-05-18 01:33:05,458 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,459 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185170_0001_1_01_000000_0] started. Is using containerId: [container_1463535185170_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D751A2136EF14677A14F40147006203E, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, startTime=1463535185459, containerId=container_1463535185170_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:05,459 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,460 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,461 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,461 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,461 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, VertexName: D751A2136EF14677A14F40147006203E, VertexParallelism: 1, TaskAttemptID:attempt_1463535185170_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B2509D7F08954CF79D5A0301A1C4F73E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:05,462 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,462 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185170_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,462 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,462 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,463 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185170_0001_00_000001, Exiting
2016-05-18 01:33:05,463 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,463 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,466 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535185170_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,466 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185170_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:05,469 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185170_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,469 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:05,469 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185170_0001_00_000001 completed successfully
2016-05-18 01:33:05,472 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,480 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D751A2136EF14677A14F40147006203E, ordinal: 1
2016-05-18 01:33:05,480 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D751A2136EF14677A14F40147006203E, mem on start (mb), free: 157, total: 395, max: 672
2016-05-18 01:33:05,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,480 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,482 INFO  output.MROutput (MROutput.java:initialize(421)) - B2509D7F08954CF79D5A0301A1C4F73E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:05,483 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,483 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B2509D7F08954CF79D5A0301A1C4F73E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [A1C305A30A0A4645A7EB63A6FAD6C8DE:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:05,483 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,483 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,483 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:05,484 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B2509D7F08954CF79D5A0301A1C4F73E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [A1C305A30A0A4645A7EB63A6FAD6C8DE:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:05,484 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,484 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A1C305A30A0A4645A7EB63A6FAD6C8DE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,484 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,486 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,485 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:05,486 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:05,487 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A1C305A30A0A4645A7EB63A6FAD6C8DE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:05,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,490 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,492 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:05,495 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:05,495 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:05,496 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535185170_0001_1_00_000000_0_10050} to DISK_DIRECT, csize=57, dsize=53, EndTime=1463535185495, TimeTaken=3, Rate=0.02 MB/s
2016-05-18 01:33:05,496 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,496 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:05,496 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A1C305A30A0A4645A7EB63A6FAD6C8DE, wasInterrupted=false
2016-05-18 01:33:05,497 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:05,497 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'insert']]
2016-05-18 01:33:05,498 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:05,498 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'insert']] streamed: true, id: C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,498 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-local-dir/output/attempt_1463535185170_0001_1_00_000000_0_10050/file.out, len=57, isLocal=true
2016-05-18 01:33:05,498 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"], id: B2509D7F08954CF79D5A0301A1C4F73E
2016-05-18 01:33:05,498 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 57 bytes from disk
2016-05-18 01:33:05,499 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:05,499 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:05,499 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:05,499 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 33 bytes
2016-05-18 01:33:05,499 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,499 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Shuffle Runner thread complete
2016-05-18 01:33:05,501 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"]
2016-05-18 01:33:05,502 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'insert']] C7B80D075E404DFAB71C3EFF8D1F2B21, for 1 inputs
2016-05-18 01:33:05,502 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D751A2136EF14677A14F40147006203E, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,503 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:05,506 INFO  output.MROutput (MROutput.java:close(526)) - B2509D7F08954CF79D5A0301A1C4F73E closed
2016-05-18 01:33:05,507 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535185170_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:05,508 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,508 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D751A2136EF14677A14F40147006203E, mem on close (mb), free: 154, total: 395, max: 672
2016-05-18 01:33:05,508 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,509 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:05,509 INFO  output.MROutput (MROutput.java:close(526)) - B2509D7F08954CF79D5A0301A1C4F73E closed
2016-05-18 01:33:05,509 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,510 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:05,511 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185170_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:05,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185170_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=10]]
2016-05-18 01:33:05,513 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185170_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:05,513 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,513 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, creationTime=1463535185456, allocationTime=1463535185457, startTime=1463535185459, finishTime=1463535185513, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,513 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,514 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, taskId=task_1463535185170_0001_1_01_000000, startTime=1463535185459, finishTime=1463535185513, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463535185170_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10
2016-05-18 01:33:05,514 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,514 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,513 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:05,515 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,515 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, vertexId=vertex_1463535185170_0001_1_01, initRequestedTime=1463535185360, initedTime=1463535185366, startRequestedTime=1463535185378, startedTime=1463535185380, finishTime=1463535185514, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535185459, firstTasksToStart=[ task_1463535185170_0001_1_01_000000 ], lastTaskFinishTime=1463535185513, lastTasksToFinish=[ task_1463535185170_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185170_0001_1_01_000000 ], longestDurationTasks=[ task_1463535185170_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,516 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,516 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185170_0001_00_000002, stoppedTime=1463535185516, exitStatus=0
2016-05-18 01:33:05,516 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:05,517 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,517 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,517 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535185170_0001_1
2016-05-18 01:33:05,517 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535185170_0001_1
2016-05-18 01:33:05,517 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: B2509D7F08954CF79D5A0301A1C4F73E for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], outputName: B2509D7F08954CF79D5A0301A1C4F73E
2016-05-18 01:33:05,517 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:05,518 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185170_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,526 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185170_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,526 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,526 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185170_0001_00_000002, Exiting
2016-05-18 01:33:05,526 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,526 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,527 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185170_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,527 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185170_0001_00_000002 completed successfully
2016-05-18 01:33:05,529 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:B2509D7F08954CF79D5A0301A1C4F73E of vertex/vertexGroup:D751A2136EF14677A14F40147006203E isVertexGroupOutput:false
2016-05-18 01:33:05,529 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,531 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535185170_0001_1, queueSize=0
2016-05-18 01:33:05,532 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535185170_0001_1, startTime=1463535185358, finishTime=1463535185529, timeTaken=171, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=340, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070370936, Process_Duration=7, Process_End_Time=2927070370943, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:33:05,532 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535185170_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:05,532 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:05 Completed Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05,532 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535185170_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:05,533 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185170_0001_1_post
2016-05-18 01:33:05,533 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,533 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:05,533 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:05,533 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, with id=dag_1463535185170_0001_1
2016-05-18 01:33:05,533 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, with id=dag_1463535185170_0001_1
2016-05-18 01:33:05,541 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001
2016-05-18 01:33:05,543 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:05,543 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:05,543 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:05,543 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:05,543 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:05,544 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:05,544 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:05,545 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:05,545 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:05,545 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,546 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:05,546 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:05,546 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,546 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:05,550 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001
2016-05-18 01:33:05,550 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified/_temporary
2016-05-18 01:33:05,551 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.401
2016-05-18 01:33:05,598 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified
2016-05-18 01:33:05,627 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:05,627 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:05,671 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:05,681 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 01:33:05,681 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:05,682 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:05,682 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:05,697 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:05,698 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown
2016-05-18 01:33:05,708 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:05,708 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,709 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"]
2016-05-18 01:33:05,709 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:05,709 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:05,710 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:05,711 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,711 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:05,725 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:05,729 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,729 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:05,729 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:05,738 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001 doesn't exist and is created
2016-05-18 01:33:05,758 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:05,758 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:05,759 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd
2016-05-18 01:33:05,775 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535185729_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,776 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,776 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,778 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:05,779 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:05,781 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:05,781 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:05,782 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:05,782 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:05,782 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:05,782 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:05,783 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:05,783 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:05,783 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,783 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,784 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:05,784 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:05,786 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:05,787 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:05,787 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:05,787 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir
2016-05-18 01:33:05,788 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/history.txt.appattempt_1463535185729_0001_000000, maxErrors=10
2016-05-18 01:33:05,788 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:05,789 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535185729_0001_000000, appSubmitTime=1463535185775, launchTime=1463535185775
2016-05-18 01:33:05,793 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:05,794 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39324
2016-05-18 01:33:05,796 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535185729
2016-05-18 01:33:05,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535185729_0001_000000, startTime=1463535185798
2016-05-18 01:33:05,798 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:05,859 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:05,859 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:05,859 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001, dagName=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,876 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,877 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:05,877 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535185729_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/dag_1463535185729_0001_1.dot
2016-05-18 01:33:05,878 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/dag_1463535185729_0001_1-tez-dag.pb.txt
2016-05-18 01:33:05,881 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185729_0001_1
2016-05-18 01:33:05,881 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,883 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05 Running Dag: dag_1463535185729_0001_1
2016-05-18 01:33:05,899 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535185729_0001_1, submitTime=1463535185877
2016-05-18 01:33:05,900 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001/tez-conf.pb" } size: 40982 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,900 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,900 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,900 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,900 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:05,901 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535185729_0001_1, initTime=1463535185901
2016-05-18 01:33:05,901 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:05,901 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:05,901 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_STARTED]: dagID=dag_1463535185729_0001_1, startTime=1463535185901
2016-05-18 01:33:05,902 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BB201390967C4EB78762D0F9F35C1925 : {A96177E652AB4CA2BA9300C2196C9610={InputName=A96177E652AB4CA2BA9300C2196C9610}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: A96177E652AB4CA2BA9300C2196C9610, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,903 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:05,903 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: A96177E652AB4CA2BA9300C2196C9610 on vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,904 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001, dagName=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,906 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input A96177E652AB4CA2BA9300C2196C9610 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:05,908 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:05,909 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:05,909 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:05,909 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:05,910 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: A96177E652AB4CA2BA9300C2196C9610 on vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,911 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,911 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {A96177E652AB4CA2BA9300C2196C9610=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:05,912 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] parallelism set to 1
2016-05-18 01:33:05,913 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,913 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], numAdditionalOutputs=1
2016-05-18 01:33:05,913 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A4F9C453867E471B91CE1103B468554C, vertex=vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:05,915 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BB201390967C4EB78762D0F9F35C1925:A4F9C453867E471B91CE1103B468554C using oldmapred API
2016-05-18 01:33:05,915 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A4F9C453867E471B91CE1103B468554C, vertexName=BB201390967C4EB78762D0F9F35C1925, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:05,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BB201390967C4EB78762D0F9F35C1925, vertexId=vertex_1463535185729_0001_1_00, initRequestedTime=1463535185902, initedTime=1463535185913, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:05,916 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] with distanceFromRoot: 0
2016-05-18 01:33:05,916 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:05,917 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185729_0001_1_00, startRequestedTime=1463535185903, startedTime=1463535185917
2016-05-18 01:33:05,917 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185729_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:05,917 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,918 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BB201390967C4EB78762D0F9F35C1925
2016-05-18 01:33:05,918 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,918 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_STARTED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskId=task_1463535185729_0001_1_00_000000, scheduledTime=1463535185918, launchTime=1463535185918
2016-05-18 01:33:05,919 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,920 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:05,922 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185729_0001_00_000001, launchTime=1463535185921
2016-05-18 01:33:05,922 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535185729_0001, with dag id: dag_1463535185729_0001_1
2016-05-18 01:33:05,923 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,924 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185729_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,924 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185729_0001_1_00_000000_0] started. Is using containerId: [container_1463535185729_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,924 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, startTime=1463535185924, containerId=container_1463535185729_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,925 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,926 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,926 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,926 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, VertexName: BB201390967C4EB78762D0F9F35C1925, VertexParallelism: 1, TaskAttemptID:attempt_1463535185729_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A96177E652AB4CA2BA9300C2196C9610, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A4F9C453867E471B91CE1103B468554C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:05,927 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,928 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535185729_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:05,928 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185729_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:05,929 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,932 INFO  input.MRInput (MRInput.java:initialize(440)) - A96177E652AB4CA2BA9300C2196C9610 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:05,932 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,933 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,943 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BB201390967C4EB78762D0F9F35C1925, ordinal: 0
2016-05-18 01:33:05,943 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB201390967C4EB78762D0F9F35C1925, mem on start (mb), free: 288, total: 394, max: 672
2016-05-18 01:33:05,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,943 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,944 INFO  output.MROutput (MROutput.java:initialize(421)) - A4F9C453867E471B91CE1103B468554C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:05,945 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,945 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A96177E652AB4CA2BA9300C2196C9610:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A4F9C453867E471B91CE1103B468554C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:05,945 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,946 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,946 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:05,946 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A96177E652AB4CA2BA9300C2196C9610:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A4F9C453867E471B91CE1103B468554C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:05,946 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A96177E652AB4CA2BA9300C2196C9610 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,947 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,948 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,949 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:05,949 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - A96177E652AB4CA2BA9300C2196C9610 initialized RecordReader from event
2016-05-18 01:33:05,951 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,952 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,952 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"], id: A4F9C453867E471B91CE1103B468554C
2016-05-18 01:33:05,952 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"]
2016-05-18 01:33:05,953 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,954 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BB201390967C4EB78762D0F9F35C1925, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,956 INFO  output.MROutput (MROutput.java:close(526)) - A4F9C453867E471B91CE1103B468554C closed
2016-05-18 01:33:05,956 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535185729_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:05,957 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,957 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB201390967C4EB78762D0F9F35C1925, mem on close (mb), free: 286, total: 394, max: 672
2016-05-18 01:33:05,957 INFO  output.MROutput (MROutput.java:close(526)) - A4F9C453867E471B91CE1103B468554C closed
2016-05-18 01:33:05,958 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,958 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:05,959 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185729_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:05,959 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185729_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:05,960 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185729_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:05,960 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,960 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:05,960 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,960 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,961 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, creationTime=1463535185919, allocationTime=1463535185921, startTime=1463535185924, finishTime=1463535185960, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,961 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskId=task_1463535185729_0001_1_00_000000, startTime=1463535185924, finishTime=1463535185962, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463535185729_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:05,963 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,963 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, vertexId=vertex_1463535185729_0001_1_00, initRequestedTime=1463535185902, initedTime=1463535185913, startRequestedTime=1463535185903, startedTime=1463535185917, finishTime=1463535185963, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535185924, firstTasksToStart=[ task_1463535185729_0001_1_00_000000 ], lastTaskFinishTime=1463535185960, lastTasksToFinish=[ task_1463535185729_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185729_0001_1_00_000000 ], longestDurationTasks=[ task_1463535185729_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,964 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,964 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185729_0001_00_000001, stoppedTime=1463535185964, exitStatus=0
2016-05-18 01:33:05,965 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:05,965 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,965 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535185729_0001_1
2016-05-18 01:33:05,965 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535185729_0001_1
2016-05-18 01:33:05,966 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A4F9C453867E471B91CE1103B468554C for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], outputName: A4F9C453867E471B91CE1103B468554C
2016-05-18 01:33:05,968 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:05,968 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185729_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,972 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185729_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,972 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185729_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,972 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185729_0001_00_000001, Exiting
2016-05-18 01:33:05,972 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,972 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,973 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185729_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,973 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185729_0001_00_000001 completed successfully
2016-05-18 01:33:05,976 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A4F9C453867E471B91CE1103B468554C of vertex/vertexGroup:BB201390967C4EB78762D0F9F35C1925 isVertexGroupOutput:false
2016-05-18 01:33:05,976 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,978 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535185729_0001_1, queueSize=0
2016-05-18 01:33:05,978 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535185729_0001_1, startTime=1463535185901, finishTime=1463535185977, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:05,978 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535185729_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:05,979 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:05 Completed Dag: dag_1463535185729_0001_1
2016-05-18 01:33:05,979 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535185729_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:05,979 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185729_0001_1_post
2016-05-18 01:33:05,979 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,979 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:05,979 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:05,980 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, with id=dag_1463535185729_0001_1
2016-05-18 01:33:05,980 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, with id=dag_1463535185729_0001_1
2016-05-18 01:33:05,986 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001
2016-05-18 01:33:05,992 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:05,992 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:05,992 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:05,992 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:05,992 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:05,993 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:05,993 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:05,993 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:05,993 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:05,994 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,994 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:05,995 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:05,995 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,996 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:06,000 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001
2016-05-18 01:33:06,001 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown/_temporary
2016-05-18 01:33:06,001 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.292
2016-05-18 01:33:06,043 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnknown/unknown
2016-05-18 01:33:06,071 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,071 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,108 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 01:33:06,117 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:06,117 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:06,117 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:06,118 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:06,135 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt
2016-05-18 01:33:06,136 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"]
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:06,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:06,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,150 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:06,165 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:06,170 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:06,170 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:06,170 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:06,179 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001 doesn't exist and is created
2016-05-18 01:33:06,200 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:06,200 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:06,201 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd
2016-05-18 01:33:06,216 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535186170_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:06,216 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:06,216 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:06,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:06,219 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:06,221 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:06,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:06,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:06,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:06,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:06,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:06,222 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:06,224 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:06,225 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:06,225 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:06,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir
2016-05-18 01:33:06,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/history.txt.appattempt_1463535186170_0001_000000, maxErrors=10
2016-05-18 01:33:06,226 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:06,227 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535186170_0001_000000, appSubmitTime=1463535186215, launchTime=1463535186215
2016-05-18 01:33:06,234 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:06,239 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36006
2016-05-18 01:33:06,239 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535186170
2016-05-18 01:33:06,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535186170_0001_000000, startTime=1463535186241
2016-05-18 01:33:06,241 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:06,301 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:06,301 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:06,301 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001, dagName=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,324 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,324 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:06,325 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535186170_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/dag_1463535186170_0001_1.dot
2016-05-18 01:33:06,326 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/dag_1463535186170_0001_1-tez-dag.pb.txt
2016-05-18 01:33:06,331 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535186170_0001_1
2016-05-18 01:33:06,331 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,331 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06 Running Dag: dag_1463535186170_0001_1
2016-05-18 01:33:06,347 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535186170_0001_1, submitTime=1463535186324
2016-05-18 01:33:06,348 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001/tez-conf.pb" } size: 40982 timestamp: 1463535186000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:06,348 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,349 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,349 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:06,349 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:06,349 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535186170_0001_1, initTime=1463535186349
2016-05-18 01:33:06,350 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:06,350 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:06,351 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_STARTED]: dagID=dag_1463535186170_0001_1, startTime=1463535186351
2016-05-18 01:33:06,351 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:06,351 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A311984AD1EE4E8FB13476FDB7F938DD : {572DB052DBB5491792851F935C3301F1={InputName=572DB052DBB5491792851F935C3301F1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:06,351 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 572DB052DBB5491792851F935C3301F1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:06,351 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,351 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,351 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,352 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,353 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:06,354 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 572DB052DBB5491792851F935C3301F1 on vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,357 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 572DB052DBB5491792851F935C3301F1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:06,359 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:06,359 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 828 Min splitLength: 52428800 New desired splits: 1 Total length: 828 Original splits: 2
2016-05-18 01:33:06,359 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 828 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 828 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:06,360 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:06,360 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 572DB052DBB5491792851F935C3301F1 on vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,362 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:06,362 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001, dagName=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,362 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {572DB052DBB5491792851F935C3301F1=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:06,364 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] parallelism set to 1
2016-05-18 01:33:06,365 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:06,365 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], numAdditionalOutputs=1
2016-05-18 01:33:06,365 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=DFA8E854335A4C1DB3A5A4F7AB304EE3, vertex=vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:06,367 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A311984AD1EE4E8FB13476FDB7F938DD:DFA8E854335A4C1DB3A5A4F7AB304EE3 using oldmapred API
2016-05-18 01:33:06,367 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=DFA8E854335A4C1DB3A5A4F7AB304EE3, vertexName=A311984AD1EE4E8FB13476FDB7F938DD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:06,368 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, vertexId=vertex_1463535186170_0001_1_00, initRequestedTime=1463535186351, initedTime=1463535186365, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:06,368 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] with distanceFromRoot: 0
2016-05-18 01:33:06,368 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:06,368 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535186170_0001_1_00, startRequestedTime=1463535186353, startedTime=1463535186368
2016-05-18 01:33:06,368 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535186170_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:06,368 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:06,373 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A311984AD1EE4E8FB13476FDB7F938DD
2016-05-18 01:33:06,373 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,373 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_STARTED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskId=task_1463535186170_0001_1_00_000000, scheduledTime=1463535186373, launchTime=1463535186373
2016-05-18 01:33:06,374 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,380 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:06,383 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535186170_0001_00_000001, launchTime=1463535186382
2016-05-18 01:33:06,383 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,385 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535186170_0001_1_00_000000_0] started. Is using containerId: [container_1463535186170_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:06,385 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, startTime=1463535186385, containerId=container_1463535186170_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:06,386 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535186170_0001, with dag id: dag_1463535186170_0001_1
2016-05-18 01:33:06,387 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535186170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06 Starting to run new task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,388 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,388 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,388 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:06,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, VertexName: A311984AD1EE4E8FB13476FDB7F938DD, VertexParallelism: 1, TaskAttemptID:attempt_1463535186170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=572DB052DBB5491792851F935C3301F1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DFA8E854335A4C1DB3A5A4F7AB304EE3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:06,388 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:06,394 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535186170_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:06,395 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,395 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535186170_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:06,399 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:06,400 INFO  input.MRInput (MRInput.java:initialize(440)) - 572DB052DBB5491792851F935C3301F1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:06,400 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,410 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, ordinal: 0
2016-05-18 01:33:06,410 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, mem on start (mb), free: 209, total: 394, max: 672
2016-05-18 01:33:06,411 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:06,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:06,412 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:06,418 INFO  output.MROutput (MROutput.java:initialize(421)) - DFA8E854335A4C1DB3A5A4F7AB304EE3: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:06,418 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:06,418 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[572DB052DBB5491792851F935C3301F1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DFA8E854335A4C1DB3A5A4F7AB304EE3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:06,418 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:06,418 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:06,419 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:06,419 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[572DB052DBB5491792851F935C3301F1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DFA8E854335A4C1DB3A5A4F7AB304EE3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:06,419 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:06,419 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 572DB052DBB5491792851F935C3301F1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:06,419 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,419 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:06,421 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:06,421 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:06,421 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,424 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt:0+414, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt:414+414], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=828}
2016-05-18 01:33:06,425 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 572DB052DBB5491792851F935C3301F1 initialized RecordReader from event
2016-05-18 01:33:06,425 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,426 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"] streamed: true, id: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,426 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"], id: DFA8E854335A4C1DB3A5A4F7AB304EE3
2016-05-18 01:33:06,426 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"]
2016-05-18 01:33:06,426 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,427 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:06,429 INFO  output.MROutput (MROutput.java:close(526)) - DFA8E854335A4C1DB3A5A4F7AB304EE3 closed
2016-05-18 01:33:06,429 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535186170_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:06,430 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,430 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, mem on close (mb), free: 207, total: 394, max: 672
2016-05-18 01:33:06,430 INFO  output.MROutput (MROutput.java:close(526)) - DFA8E854335A4C1DB3A5A4F7AB304EE3 closed
2016-05-18 01:33:06,430 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:06 Completed running task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,431 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:06,431 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535186170_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:06,432 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535186170_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7][cascading.flow.SliceCounters Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=7, Tuples_Written=7]]
2016-05-18 01:33:06,433 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535186170_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:06,433 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:06,433 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:06,433 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,434 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, creationTime=1463535186374, allocationTime=1463535186382, startTime=1463535186385, finishTime=1463535186434, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:06,434 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,435 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskId=task_1463535186170_0001_1_00_000000, startTime=1463535186385, finishTime=1463535186435, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463535186170_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7
2016-05-18 01:33:06,436 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:06,436 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,437 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, vertexId=vertex_1463535186170_0001_1_00, initRequestedTime=1463535186351, initedTime=1463535186365, startRequestedTime=1463535186353, startedTime=1463535186368, finishTime=1463535186436, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7, vertexStats=firstTaskStartTime=1463535186385, firstTasksToStart=[ task_1463535186170_0001_1_00_000000 ], lastTaskFinishTime=1463535186434, lastTasksToFinish=[ task_1463535186170_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535186170_0001_1_00_000000 ], longestDurationTasks=[ task_1463535186170_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:06,437 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:06,437 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535186170_0001_00_000001, stoppedTime=1463535186437, exitStatus=0
2016-05-18 01:33:06,437 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:06,437 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:06,438 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535186170_0001_1
2016-05-18 01:33:06,438 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535186170_0001_1
2016-05-18 01:33:06,438 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:06,438 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: DFA8E854335A4C1DB3A5A4F7AB304EE3 for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], outputName: DFA8E854335A4C1DB3A5A4F7AB304EE3
2016-05-18 01:33:06,438 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535186170_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:06,439 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,439 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535186170_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:06,439 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535186170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:06,439 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535186170_0001_00_000001, Exiting
2016-05-18 01:33:06,440 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,440 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,442 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535186170_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:06,443 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535186170_0001_00_000001 completed successfully
2016-05-18 01:33:06,449 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:DFA8E854335A4C1DB3A5A4F7AB304EE3 of vertex/vertexGroup:A311984AD1EE4E8FB13476FDB7F938DD isVertexGroupOutput:false
2016-05-18 01:33:06,449 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:06,450 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535186170_0001_1, queueSize=0
2016-05-18 01:33:06,450 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535186170_0001_1, startTime=1463535186350, finishTime=1463535186449, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=280, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7
2016-05-18 01:33:06,451 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535186170_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:06,451 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:06 Completed Dag: dag_1463535186170_0001_1
2016-05-18 01:33:06,451 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535186170_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:06,451 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535186170_0001_1_post
2016-05-18 01:33:06,451 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,451 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:06,452 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:06,452 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, with id=dag_1463535186170_0001_1
2016-05-18 01:33:06,452 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, with id=dag_1463535186170_0001_1
2016-05-18 01:33:06,464 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001
2016-05-18 01:33:06,466 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:06,466 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:06,466 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:06,466 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:06,467 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:06,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:06,467 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:06,467 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:06,470 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:06,470 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:06,470 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:06,470 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:06,471 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:06,471 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:06,475 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001
2016-05-18 01:33:06,475 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth/_temporary
2016-05-18 01:33:06,476 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.326
2016-05-18 01:33:06,520 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth
2016-05-18 01:33:06,547 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,547 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,575 WARN  planner.FlowPlanner (FlowPlanner.java:verifyPipeAssemblyEndPoints(464)) - duplicate head name found, not an error but heads should have unique names: 'test'
2016-05-18 01:33:06,575 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDupeHeadNames/unknown
2016-05-18 01:33:06,611 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,612 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,640 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDupeTailNames/unknown
2016-05-18 01:33:06,698 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:06,698 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:06,952 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,953 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:07,212 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:07,212 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:07,658 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:07,658 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:08,150 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:08,150 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:08,690 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:08,690 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:09,170 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:09,170 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:09,176 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:02.174
2016-05-18 01:33:09,308 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:02.298
2016-05-18 01:33:09,312 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:09,313 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:09,313 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:09,418 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt
2016-05-18 01:33:09,569 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] starting
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"]
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  parallel execution of steps is enabled: false
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  executing total steps: 1
2016-05-18 01:33:09,730 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  allocating management threads: 1
2016-05-18 01:33:09,731 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] starting step: (1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,731 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [pipeline] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:09,745 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tez session mode enabled: true
2016-05-18 01:33:09,749 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:09,750 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:09,750 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:09,760 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001 doesn't exist and is created
2016-05-18 01:33:09,780 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:09,780 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:09,781 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd
2016-05-18 01:33:09,798 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535189750_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:09,798 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:09,799 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:09,801 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:09,801 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:09,801 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:09,802 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:09,803 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:09,807 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:09,808 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:09,808 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:09,809 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:09,809 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:09,809 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:09,811 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:09,811 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:09,811 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:09,811 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:09,813 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:09,813 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:09,815 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:09,816 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:09,816 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:09,817 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir
2016-05-18 01:33:09,817 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/history.txt.appattempt_1463535189750_0001_000000, maxErrors=10
2016-05-18 01:33:09,817 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:09,818 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535189750_0001_000000, appSubmitTime=1463535189797, launchTime=1463535189797
2016-05-18 01:33:09,821 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:09,827 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48088
2016-05-18 01:33:09,827 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535189750
2016-05-18 01:33:09,828 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535189750_0001_000000, startTime=1463535189828
2016-05-18 01:33:09,829 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:09,880 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:09,881 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:09,881 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001, dagName=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,898 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,899 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:09,900 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535189750_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/dag_1463535189750_0001_1.dot
2016-05-18 01:33:09,901 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/dag_1463535189750_0001_1-tez-dag.pb.txt
2016-05-18 01:33:09,910 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535189750_0001_1
2016-05-18 01:33:09,910 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:09,911 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09 Running Dag: dag_1463535189750_0001_1
2016-05-18 01:33:09,925 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535189750_0001_1, submitTime=1463535189898
2016-05-18 01:33:09,926 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/tez-conf.pb" } size: 40938 timestamp: 1463535189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:09,927 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,927 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:09,928 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/tez-conf.pb" } size: 40938 timestamp: 1463535189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:09,928 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,928 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:09,928 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:09,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535189750_0001_1, initTime=1463535189929
2016-05-18 01:33:09,929 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:09,930 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:09,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_STARTED]: dagID=dag_1463535189750_0001_1, startTime=1463535189930
2016-05-18 01:33:09,930 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:09,930 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B40B2F7CFAB54DCD978B10E5748D3937 : {23A788B655194A789395FF318DE1A1C9={InputName=23A788B655194A789395FF318DE1A1C9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:09,930 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 23A788B655194A789395FF318DE1A1C9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:09,930 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,932 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001, dagName=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,934 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:09,934 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 23A788B655194A789395FF318DE1A1C9 on vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,934 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,937 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 23A788B655194A789395FF318DE1A1C9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:09,937 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:09,937 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,938 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,938 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:09,939 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 32599 Min splitLength: 52428800 New desired splits: 1 Total length: 32599 Original splits: 2
2016-05-18 01:33:09,939 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 32599 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 32599 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:09,939 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:09,940 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535189750_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:09,940 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], numAdditionalOutputs=1
2016-05-18 01:33:09,940 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F56003740507449D974ACBE04D44D7F9, vertex=vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:09,940 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 23A788B655194A789395FF318DE1A1C9 on vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,943 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FBF8388363404EFBBC6F45982472C1A1:F56003740507449D974ACBE04D44D7F9 using oldmapred API
2016-05-18 01:33:09,943 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F56003740507449D974ACBE04D44D7F9, vertexName=FBF8388363404EFBBC6F45982472C1A1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:09,944 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, vertexId=vertex_1463535189750_0001_1_01, initRequestedTime=1463535189934, initedTime=1463535189940, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:09,945 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:09,946 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:09,946 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {23A788B655194A789395FF318DE1A1C9=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:09,947 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] parallelism set to 1
2016-05-18 01:33:09,947 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:09,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, vertexId=vertex_1463535189750_0001_1_00, initRequestedTime=1463535189930, initedTime=1463535189947, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:09,948 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] with distanceFromRoot: 0
2016-05-18 01:33:09,948 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:09,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535189750_0001_1_00, startRequestedTime=1463535189934, startedTime=1463535189948
2016-05-18 01:33:09,949 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535189750_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:09,949 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:09,949 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535189750_0001_1_00 for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] numStartedSources: 1 numSources: 1
2016-05-18 01:33:09,949 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:09,949 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,949 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] with distanceFromRoot: 1
2016-05-18 01:33:09,952 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535189750_0001_1_01, startRequestedTime=1463535189949, startedTime=1463535189952
2016-05-18 01:33:09,952 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:09,954 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: FBF8388363404EFBBC6F45982472C1A1 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:09,957 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_STARTED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskId=task_1463535189750_0001_1_00_000000, scheduledTime=1463535189957, launchTime=1463535189957
2016-05-18 01:33:09,958 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,958 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B40B2F7CFAB54DCD978B10E5748D3937 in vertex: FBF8388363404EFBBC6F45982472C1A1 numBipartiteSourceTasks: 1
2016-05-18 01:33:09,959 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] submitted tez dag to app master: application_1463535189750_0001, with dag id: dag_1463535189750_0001_1
2016-05-18 01:33:09,959 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:09,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535189750_0001_00_000001, launchTime=1463535189962
2016-05-18 01:33:09,964 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000001
2016-05-18 01:33:09,966 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535189750_0001_1_00_000000_0] started. Is using containerId: [container_1463535189750_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:09,966 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, startTime=1463535189965, containerId=container_1463535189750_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:09,966 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09 Starting to run new task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,967 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,967 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:09,968 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:09,968 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, VertexName: B40B2F7CFAB54DCD978B10E5748D3937, VertexParallelism: 1, TaskAttemptID:attempt_1463535189750_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23A788B655194A789395FF318DE1A1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FBF8388363404EFBBC6F45982472C1A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:09,969 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:09,969 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,970 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535189750_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:09,971 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535189750_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:09,974 INFO  input.MRInput (MRInput.java:initialize(440)) - 23A788B655194A789395FF318DE1A1C9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:09,974 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:09,975 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:09,978 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:09,979 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:10,100 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, ordinal: 0
2016-05-18 01:33:10,100 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, mem on start (mb), free: 290, total: 394, max: 672
2016-05-18 01:33:10,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:10,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:10,100 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:10,101 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:10,101 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[23A788B655194A789395FF318DE1A1C9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FBF8388363404EFBBC6F45982472C1A1:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:10,102 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:10,102 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:10,103 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:10,103 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[23A788B655194A789395FF318DE1A1C9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FBF8388363404EFBBC6F45982472C1A1:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:10,103 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:10,103 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 23A788B655194A789395FF318DE1A1C9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:10,103 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:10,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:10,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:10,105 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:10,105 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,107 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt:0+16299, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt:16299+16300], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=32599}
2016-05-18 01:33:10,107 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 23A788B655194A789395FF318DE1A1C9 initialized RecordReader from event
2016-05-18 01:33:10,108 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:10,137 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"] streamed: true, id: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:10,137 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(pipeline)[by:[{1}:'hash']], id: E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,139 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(pipeline)[by:[{1}:'hash']] E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,140 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - FBF8388363404EFBBC6F45982472C1A1 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5a65f83f, comparator=cascading.tuple.hadoop.util.TupleComparator@5c8d1e76, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:10,141 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for FBF8388363404EFBBC6F45982472C1A1: , UsingHashComparator=false
2016-05-18 01:33:10,155 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:10,155 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:10,155 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for FBF8388363404EFBBC6F45982472C1A1: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:10,155 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - FBF8388363404EFBBC6F45982472C1A1: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:10,236 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:10,236 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:10,362 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,363 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, mem on close (mb), free: 176, total: 394, max: 672
2016-05-18 01:33:10,363 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - FBF8388363404EFBBC6F45982472C1A1: Starting flush of map output
2016-05-18 01:33:10,363 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - FBF8388363404EFBBC6F45982472C1A1: Span0.length = 200, perItem = 176
2016-05-18 01:33:10,385 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - FBF8388363404EFBBC6F45982472C1A1: done sorting span=0, length=200, time=22
2016-05-18 01:33:10,386 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - FBF8388363404EFBBC6F45982472C1A1: Heap = SpanIterator<0:199> (span=Span[16777216,35385]),
2016-05-18 01:33:10,387 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - FBF8388363404EFBBC6F45982472C1A1: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-local-dir/output/attempt_1463535189750_0001_1_00_000000_0_10062_0/file.out
2016-05-18 01:33:10,394 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,394 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:10,395 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535189750_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:10,396 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535189750_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27][cascading.flow.StepCounters Tuples_Read=200]]
2016-05-18 01:33:10,398 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535189750_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:10,398 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,398 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:10,398 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,398 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,398 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, creationTime=1463535189957, allocationTime=1463535189961, startTime=1463535189965, finishTime=1463535190398, timeTaken=433, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:10,399 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,401 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskId=task_1463535189750_0001_1_00_000000, startTime=1463535189965, finishTime=1463535190399, timeTaken=434, status=SUCCEEDED, successfulAttemptID=attempt_1463535189750_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27, cascading.flow.StepCounters, Tuples_Read=200
2016-05-18 01:33:10,401 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:10,401 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:10,404 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, vertexId=vertex_1463535189750_0001_1_00, initRequestedTime=1463535189930, initedTime=1463535189947, startRequestedTime=1463535189934, startedTime=1463535189948, finishTime=1463535190401, timeTaken=453, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1463535189965, firstTasksToStart=[ task_1463535189750_0001_1_00_000000 ], lastTaskFinishTime=1463535190398, lastTasksToFinish=[ task_1463535189750_0001_1_00_000000 ], minTaskDuration=433, maxTaskDuration=433, avgTaskDuration=433.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535189750_0001_1_00_000000 ], longestDurationTasks=[ task_1463535189750_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:10,404 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:10,404 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535189750_0001_00_000001, stoppedTime=1463535190404, exitStatus=0
2016-05-18 01:33:10,404 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] attempt: attempt_1463535189750_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:10,405 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:10,405 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FBF8388363404EFBBC6F45982472C1A1
2016-05-18 01:33:10,405 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,405 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B40B2F7CFAB54DCD978B10E5748D3937, destinationVertexName=FBF8388363404EFBBC6F45982472C1A1. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:10,405 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,405 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535189750_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:10,406 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_STARTED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskId=task_1463535189750_0001_1_01_000000, scheduledTime=1463535190406, launchTime=1463535190406
2016-05-18 01:33:10,406 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,408 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535189750_0001_00_000002, launchTime=1463535190408
2016-05-18 01:33:10,408 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,409 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,409 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535189750_0001_1_01_000000_0] started. Is using containerId: [container_1463535189750_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:10 Starting to run new task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,410 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,410 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,410 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:10,411 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, VertexName: FBF8388363404EFBBC6F45982472C1A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535189750_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B40B2F7CFAB54DCD978B10E5748D3937, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F56003740507449D974ACBE04D44D7F9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:10,411 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:10,412 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,412 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535189750_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,409 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535189750_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:10,411 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, startTime=1463535190409, containerId=container_1463535189750_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:10,414 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000001: 16 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:10,415 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:10,415 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535189750_0001_00_000001, Exiting
2016-05-18 01:33:10,415 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,415 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,420 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535189750_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:10,421 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535189750_0001_00_000001 completed successfully
2016-05-18 01:33:10,422 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535189750_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:10,424 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:10,432 INFO  output.MROutput (MROutput.java:initialize(421)) - F56003740507449D974ACBE04D44D7F9: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:10,511 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, ordinal: 1
2016-05-18 01:33:10,512 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, mem on start (mb), free: 168, total: 394, max: 672
2016-05-18 01:33:10,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:10,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:10,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:10,512 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:10,512 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B40B2F7CFAB54DCD978B10E5748D3937:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [F56003740507449D974ACBE04D44D7F9:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:10,512 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:10,513 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:10,513 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:10,513 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B40B2F7CFAB54DCD978B10E5748D3937:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [F56003740507449D974ACBE04D44D7F9:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:10,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:10,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B40B2F7CFAB54DCD978B10E5748D3937 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:10,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:10,513 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:10,514 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B40B2F7CFAB54DCD978B10E5748D3937: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:10,514 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B40B2F7CFAB54DCD978B10E5748D3937: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:10,516 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B40B2F7CFAB54DCD978B10E5748D3937 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:10,516 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,516 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:10,517 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,518 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B40B2F7CFAB54DCD978B10E5748D3937: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:10,521 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(pipeline)[by:[{1}:'hash']]
2016-05-18 01:33:10,524 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535189750_0001_1_00_000000_0_10062} to DISK_DIRECT, csize=35992, dsize=35988, EndTime=1463535190524, TimeTaken=5, Rate=6.86 MB/s
2016-05-18 01:33:10,524 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,524 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.03 MB/s)
2016-05-18 01:33:10,525 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B40B2F7CFAB54DCD978B10E5748D3937, wasInterrupted=false
2016-05-18 01:33:10,526 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.03 MB/s)
2016-05-18 01:33:10,526 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - B40B2F7CFAB54DCD978B10E5748D3937: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:10,526 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:10,527 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-local-dir/output/attempt_1463535189750_0001_1_00_000000_0_10062/file.out, len=35992, isLocal=true
2016-05-18 01:33:10,527 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 35992 bytes from disk
2016-05-18 01:33:10,527 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:10,527 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:10,528 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 35975 bytes
2016-05-18 01:33:10,528 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,528 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B40B2F7CFAB54DCD978B10E5748D3937: Shuffle Runner thread complete
2016-05-18 01:33:10,546 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(pipeline)[by:[{1}:'hash']] streamed: true, id: E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,546 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"], id: F56003740507449D974ACBE04D44D7F9
2016-05-18 01:33:10,547 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:10,547 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:10,548 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"]
2016-05-18 01:33:10,549 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(pipeline)[by:[{1}:'hash']] E6438AF0FDD8483EB8A88D81E849C72A, for 1 inputs
2016-05-18 01:33:10,549 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:10,550 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B40B2F7CFAB54DCD978B10E5748D3937: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:10,916 INFO  output.MROutput (MROutput.java:close(526)) - F56003740507449D974ACBE04D44D7F9 closed
2016-05-18 01:33:10,917 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535189750_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:10,918 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,918 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, mem on close (mb), free: 154, total: 394, max: 672
2016-05-18 01:33:10,919 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,919 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B40B2F7CFAB54DCD978B10E5748D3937: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:10,919 INFO  output.MROutput (MROutput.java:close(526)) - F56003740507449D974ACBE04D44D7F9 closed
2016-05-18 01:33:10,920 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,921 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:10,922 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535189750_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:10,923 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535189750_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3][cascading.flow.StepCounters Tuples_Written=200]]
2016-05-18 01:33:10,925 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535189750_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:10,925 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,926 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:10,926 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,926 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,926 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, creationTime=1463535190406, allocationTime=1463535190407, startTime=1463535190409, finishTime=1463535190926, timeTaken=517, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:10,927 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,928 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskId=task_1463535189750_0001_1_01_000000, startTime=1463535190409, finishTime=1463535190927, timeTaken=518, status=SUCCEEDED, successfulAttemptID=attempt_1463535189750_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=200
2016-05-18 01:33:10,928 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:10,928 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, vertexId=vertex_1463535189750_0001_1_01, initRequestedTime=1463535189934, initedTime=1463535189940, startRequestedTime=1463535189949, startedTime=1463535189952, finishTime=1463535190928, timeTaken=976, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=200, vertexStats=firstTaskStartTime=1463535190409, firstTasksToStart=[ task_1463535189750_0001_1_01_000000 ], lastTaskFinishTime=1463535190926, lastTasksToFinish=[ task_1463535189750_0001_1_01_000000 ], minTaskDuration=517, maxTaskDuration=517, avgTaskDuration=517.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535189750_0001_1_01_000000 ], longestDurationTasks=[ task_1463535189750_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:10,929 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:10,929 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:10,929 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,929 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:10,929 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535189750_0001_1
2016-05-18 01:33:10,929 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535189750_0001_1
2016-05-18 01:33:10,930 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:10,930 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F56003740507449D974ACBE04D44D7F9 for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], outputName: F56003740507449D974ACBE04D44D7F9
2016-05-18 01:33:10,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535189750_0001_00_000002, stoppedTime=1463535190930, exitStatus=0
2016-05-18 01:33:10,931 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535189750_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:10,931 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535189750_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:10,933 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:10,934 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535189750_0001_00_000002, Exiting
2016-05-18 01:33:10,934 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,934 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,934 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535189750_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:10,934 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535189750_0001_00_000002 completed successfully
2016-05-18 01:33:10,944 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F56003740507449D974ACBE04D44D7F9 of vertex/vertexGroup:FBF8388363404EFBBC6F45982472C1A1 isVertexGroupOutput:false
2016-05-18 01:33:10,944 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,947 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535189750_0001_1, queueSize=0
2016-05-18 01:33:10,948 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535189750_0001_1, startTime=1463535189930, finishTime=1463535190944, timeTaken=1014, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=2130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=400, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=400, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070380786, Process_Duration=494, Process_End_Time=2927070381280, Read_Duration=16, Tuples_Read=400, Tuples_Written=400, Write_Duration=30, cascading.flow.StepCounters, Tuples_Read=200, Tuples_Written=200
2016-05-18 01:33:10,949 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535189750_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:10,949 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:10 Completed Dag: dag_1463535189750_0001_1
2016-05-18 01:33:10,949 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535189750_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:10,950 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535189750_0001_1_post
2016-05-18 01:33:10,950 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,950 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:10,950 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:10,951 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, with id=dag_1463535189750_0001_1
2016-05-18 01:33:10,951 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, with id=dag_1463535189750_0001_1
2016-05-18 01:33:10,957 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001
2016-05-18 01:33:10,960 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:10,960 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:10,960 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:10,960 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:10,960 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:10,961 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:10,962 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:10,965 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:10,965 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:10,965 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:10,966 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:10,966 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:10,967 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:10,968 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:10,975 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001
2016-05-18 01:33:10,976 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline/_temporary
2016-05-18 01:33:10,977 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline]  completed in: 00:01.246
2016-05-18 01:33:10,994 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:10,994 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:11,042 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline
2016-05-18 01:33:11,080 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:11,081 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:11,116 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-18 01:33:11,124 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 01:33:11,124 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:11,125 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:11,125 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:11,141 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:33:11,141 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  parallel execution of steps is enabled: false
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  executing total steps: 1
2016-05-18 01:33:11,149 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  allocating management threads: 1
2016-05-18 01:33:11,150 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting step: (1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,150 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [split] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:11,163 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tez session mode enabled: true
2016-05-18 01:33:11,167 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,168 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:11,168 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:11,176 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001 doesn't exist and is created
2016-05-18 01:33:11,198 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:11,198 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:11,199 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd
2016-05-18 01:33:11,214 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535191167_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,214 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,214 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,217 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:11,218 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:11,220 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:11,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:11,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:11,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:11,221 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:11,222 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:11,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:11,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:11,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,222 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,223 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:11,223 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:11,225 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:11,225 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:11,226 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:11,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir
2016-05-18 01:33:11,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/history.txt.appattempt_1463535191167_0001_000000, maxErrors=10
2016-05-18 01:33:11,226 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:11,226 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535191167_0001_000000, appSubmitTime=1463535191213, launchTime=1463535191214
2016-05-18 01:33:11,233 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:11,233 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40690
2016-05-18 01:33:11,236 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535191167
2016-05-18 01:33:11,239 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535191167_0001_000000, startTime=1463535191239
2016-05-18 01:33:11,239 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:11,299 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:11,299 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:11,299 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001, dagName=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,317 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,318 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:11,319 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535191167_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/dag_1463535191167_0001_1.dot
2016-05-18 01:33:11,319 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/dag_1463535191167_0001_1-tez-dag.pb.txt
2016-05-18 01:33:11,323 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191167_0001_1
2016-05-18 01:33:11,323 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,323 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11 Running Dag: dag_1463535191167_0001_1
2016-05-18 01:33:11,337 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535191167_0001_1, submitTime=1463535191318
2016-05-18 01:33:11,338 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:11,338 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,338 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,339 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:11,339 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:11,340 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535191167_0001_1, initTime=1463535191339
2016-05-18 01:33:11,341 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:11,341 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:11,341 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_STARTED]: dagID=dag_1463535191167_0001_1, startTime=1463535191341
2016-05-18 01:33:11,341 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001, dagName=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,342 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:11,343 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D3350753BFB44335BB67E70CE86C1D0C : {A80D293996C5484BB2BD69789D8648CA={InputName=A80D293996C5484BB2BD69789D8648CA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:11,343 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: A80D293996C5484BB2BD69789D8648CA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:11,343 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,343 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,343 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,344 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,344 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:11,345 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: A80D293996C5484BB2BD69789D8648CA on vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,347 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input A80D293996C5484BB2BD69789D8648CA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:11,349 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:11,349 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:11,349 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:11,350 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:11,350 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: A80D293996C5484BB2BD69789D8648CA on vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,351 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:11,351 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {A80D293996C5484BB2BD69789D8648CA=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:11,352 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] parallelism set to 1
2016-05-18 01:33:11,352 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:11,353 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], numAdditionalOutputs=1
2016-05-18 01:33:11,353 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=963FB9B4FA5843BBB36D210BA6B2CFED, vertex=vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:11,355 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D3350753BFB44335BB67E70CE86C1D0C:963FB9B4FA5843BBB36D210BA6B2CFED using oldmapred API
2016-05-18 01:33:11,355 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=963FB9B4FA5843BBB36D210BA6B2CFED, vertexName=D3350753BFB44335BB67E70CE86C1D0C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:11,356 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, vertexId=vertex_1463535191167_0001_1_00, initRequestedTime=1463535191343, initedTime=1463535191353, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:11,356 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] with distanceFromRoot: 0
2016-05-18 01:33:11,356 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:11,357 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191167_0001_1_00, startRequestedTime=1463535191344, startedTime=1463535191357
2016-05-18 01:33:11,357 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191167_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:11,357 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:11,357 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D3350753BFB44335BB67E70CE86C1D0C
2016-05-18 01:33:11,358 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_STARTED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskId=task_1463535191167_0001_1_00_000000, scheduledTime=1463535191359, launchTime=1463535191359
2016-05-18 01:33:11,359 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,360 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:11,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191167_0001_00_000001, launchTime=1463535191362
2016-05-18 01:33:11,362 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,363 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split] submitted tez dag to app master: application_1463535191167_0001, with dag id: dag_1463535191167_0001_1
2016-05-18 01:33:11,365 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191167_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11 Starting to run new task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,366 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,366 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,366 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:11,366 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, VertexName: D3350753BFB44335BB67E70CE86C1D0C, VertexParallelism: 1, TaskAttemptID:attempt_1463535191167_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A80D293996C5484BB2BD69789D8648CA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:11,367 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:11,367 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,368 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191167_0001_1_00_000000_0] started. Is using containerId: [container_1463535191167_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:11,368 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191167_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:11,369 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, startTime=1463535191368, containerId=container_1463535191167_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:11,373 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191167_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:11,373 INFO  input.MRInput (MRInput.java:initialize(440)) - A80D293996C5484BB2BD69789D8648CA using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:11,374 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,379 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:11,385 INFO  output.MROutput (MROutput.java:initialize(421)) - 963FB9B4FA5843BBB36D210BA6B2CFED: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:11,386 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, ordinal: 0
2016-05-18 01:33:11,386 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, mem on start (mb), free: 284, total: 394, max: 672
2016-05-18 01:33:11,386 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:11,386 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:11,387 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:11,387 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:11,387 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A80D293996C5484BB2BD69789D8648CA:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [963FB9B4FA5843BBB36D210BA6B2CFED:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:11,387 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:11,388 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:11,388 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:11,388 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A80D293996C5484BB2BD69789D8648CA:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:11,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:11,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A80D293996C5484BB2BD69789D8648CA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:11,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:11,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:11,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:11,389 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,393 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:11,393 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - A80D293996C5484BB2BD69789D8648CA initialized RecordReader from event
2016-05-18 01:33:11,393 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,394 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,394 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"], id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:11,394 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,394 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,395 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:11,398 INFO  output.MROutput (MROutput.java:close(526)) - 963FB9B4FA5843BBB36D210BA6B2CFED closed
2016-05-18 01:33:11,398 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535191167_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:11,399 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,399 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, mem on close (mb), free: 282, total: 394, max: 672
2016-05-18 01:33:11,400 INFO  output.MROutput (MROutput.java:close(526)) - 963FB9B4FA5843BBB36D210BA6B2CFED closed
2016-05-18 01:33:11,400 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:11 Completed running task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,400 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:11,400 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191167_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:11,401 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191167_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:11,402 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191167_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:11,402 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:11,402 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:11,402 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,403 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, creationTime=1463535191359, allocationTime=1463535191361, startTime=1463535191368, finishTime=1463535191402, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:11,403 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,407 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,407 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskId=task_1463535191167_0001_1_00_000000, startTime=1463535191368, finishTime=1463535191403, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463535191167_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:11,407 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:11,407 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,409 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, vertexId=vertex_1463535191167_0001_1_00, initRequestedTime=1463535191343, initedTime=1463535191353, startRequestedTime=1463535191344, startedTime=1463535191357, finishTime=1463535191407, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535191368, firstTasksToStart=[ task_1463535191167_0001_1_00_000000 ], lastTaskFinishTime=1463535191402, lastTasksToFinish=[ task_1463535191167_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191167_0001_1_00_000000 ], longestDurationTasks=[ task_1463535191167_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:11,409 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:11,409 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191167_0001_00_000001, stoppedTime=1463535191409, exitStatus=0
2016-05-18 01:33:11,409 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:11,409 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:11,410 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535191167_0001_1
2016-05-18 01:33:11,410 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535191167_0001_1
2016-05-18 01:33:11,410 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191167_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:11,411 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:11,410 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 963FB9B4FA5843BBB36D210BA6B2CFED for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], outputName: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:11,411 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191167_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:11,411 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191167_0001_00_000001, Exiting
2016-05-18 01:33:11,411 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,412 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,414 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191167_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:11,415 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191167_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:11,415 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191167_0001_00_000001 completed successfully
2016-05-18 01:33:11,422 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:963FB9B4FA5843BBB36D210BA6B2CFED of vertex/vertexGroup:D3350753BFB44335BB67E70CE86C1D0C isVertexGroupOutput:false
2016-05-18 01:33:11,422 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:11,423 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535191167_0001_1, queueSize=0
2016-05-18 01:33:11,424 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535191167_0001_1, startTime=1463535191341, finishTime=1463535191422, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:11,424 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535191167_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:11,424 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:11 Completed Dag: dag_1463535191167_0001_1
2016-05-18 01:33:11,425 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535191167_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:11,425 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191167_0001_1_post
2016-05-18 01:33:11,425 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,425 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:11,425 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:11,425 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, with id=dag_1463535191167_0001_1
2016-05-18 01:33:11,425 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, with id=dag_1463535191167_0001_1
2016-05-18 01:33:11,440 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001
2016-05-18 01:33:11,442 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:11,442 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:11,442 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:11,442 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:11,442 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:11,443 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:11,443 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:11,446 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:11,446 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:11,447 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:11,447 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:11,447 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:11,448 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:11,450 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:11,455 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001
2016-05-18 01:33:11,456 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/_temporary
2016-05-18 01:33:11,457 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [split]  completed in: 00:00.307
2016-05-18 01:33:11,471 INFO  app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:11,471 INFO  app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:11,486 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:11,487 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:11,535 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 01:33:11,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 01:33:11,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:11,543 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:11,544 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:11,561 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,566 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,572 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit
2016-05-18 01:33:11,624 INFO  tez.Hadoop2TezFlowStep (Hadoop2TezFlowStep.java:createDAG(265)) - adding grouped edge between: A24AE922279248E69EE6985AE886DB2D,FE62E2013A1A4DE5B2359A2A3196EDC1 and F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:11,627 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] starting
2016-05-18 01:33:11,627 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,628 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"]
2016-05-18 01:33:11,628 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  parallel execution of steps is enabled: false
2016-05-18 01:33:11,628 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  executing total steps: 1
2016-05-18 01:33:11,628 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  allocating management threads: 1
2016-05-18 01:33:11,629 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] starting step: (1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:11,629 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [lhs+rhs] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:11,647 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tez session mode enabled: true
2016-05-18 01:33:11,651 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,652 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:11,652 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:11,662 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001 doesn't exist and is created
2016-05-18 01:33:11,683 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:11,683 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:11,684 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd
2016-05-18 01:33:11,701 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535191652_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,702 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,702 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,704 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:11,705 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:11,707 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:11,707 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:11,708 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:11,708 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:11,708 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:11,708 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:11,709 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:11,709 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:11,709 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,709 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,709 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:11,710 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:11,712 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:11,713 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:11,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:11,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir
2016-05-18 01:33:11,714 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/history.txt.appattempt_1463535191652_0001_000000, maxErrors=10
2016-05-18 01:33:11,715 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:11,715 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535191652_0001_000000, appSubmitTime=1463535191701, launchTime=1463535191701
2016-05-18 01:33:11,721 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:11,730 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35647
2016-05-18 01:33:11,730 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535191652
2016-05-18 01:33:11,737 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535191652_0001_000000, startTime=1463535191737
2016-05-18 01:33:11,738 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:11,783 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:11,784 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:11,784 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001, dagName=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,012 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,013 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:12,013 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535191652_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/dag_1463535191652_0001_1.dot
2016-05-18 01:33:12,014 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/dag_1463535191652_0001_1-tez-dag.pb.txt
2016-05-18 01:33:12,029 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191652_0001_1
2016-05-18 01:33:12,029 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,029 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12 Running Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12,045 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535191652_0001_1, submitTime=1463535192012
2016-05-18 01:33:12,047 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,047 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,047 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,048 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,048 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,048 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,051 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,051 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,051 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,051 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:12,053 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535191652_0001_1, initTime=1463535192053
2016-05-18 01:33:12,053 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:12,054 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:12,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_STARTED]: dagID=dag_1463535191652_0001_1, startTime=1463535192054
2016-05-18 01:33:12,054 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:12,054 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FE62E2013A1A4DE5B2359A2A3196EDC1 : {963FB9B4FA5843BBB36D210BA6B2CFED={InputName=963FB9B4FA5843BBB36D210BA6B2CFED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,054 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 963FB9B4FA5843BBB36D210BA6B2CFED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,054 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,054 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,054 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,055 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A24AE922279248E69EE6985AE886DB2D : {963FB9B4FA5843BBB36D210BA6B2CFED={InputName=963FB9B4FA5843BBB36D210BA6B2CFED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,056 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 963FB9B4FA5843BBB36D210BA6B2CFED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,056 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,057 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,057 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001, dagName=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,058 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,058 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,061 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 963FB9B4FA5843BBB36D210BA6B2CFED asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,062 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,062 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,062 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,064 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,064 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], numAdditionalOutputs=1
2016-05-18 01:33:12,064 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F1CC6B9FD79F49E0BAE2E82A62FDA501, vertex=vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:12,065 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 963FB9B4FA5843BBB36D210BA6B2CFED asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,068 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F7C5BC8A38EB422FAAA1BF2B645CE780:F1CC6B9FD79F49E0BAE2E82A62FDA501 using oldmapred API
2016-05-18 01:33:12,068 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F1CC6B9FD79F49E0BAE2E82A62FDA501, vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:12,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, vertexId=vertex_1463535191652_0001_1_02, initRequestedTime=1463535192057, initedTime=1463535192064, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,074 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,081 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] submitted tez dag to app master: application_1463535191652_0001, with dag id: dag_1463535191652_0001_1
2016-05-18 01:33:12,085 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,085 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:33:12,085 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,085 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,086 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,087 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,087 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {963FB9B4FA5843BBB36D210BA6B2CFED=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,088 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] parallelism set to 1
2016-05-18 01:33:12,088 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,089 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A24AE922279248E69EE6985AE886DB2D, vertexId=vertex_1463535191652_0001_1_01, initRequestedTime=1463535192055, initedTime=1463535192089, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,089 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] with distanceFromRoot: 0
2016-05-18 01:33:12,089 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,089 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_01, startRequestedTime=1463535192057, startedTime=1463535192089
2016-05-18 01:33:12,089 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,089 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,090 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535191652_0001_1_01 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 1 numSources: 2
2016-05-18 01:33:12,090 INFO  impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 1 numSources: 2
2016-05-18 01:33:12,091 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,091 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:33:12,091 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,091 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,092 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,092 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,092 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {963FB9B4FA5843BBB36D210BA6B2CFED=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,093 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] parallelism set to 1
2016-05-18 01:33:12,093 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,093 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, vertexId=vertex_1463535191652_0001_1_00, initRequestedTime=1463535192054, initedTime=1463535192093, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,093 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] with distanceFromRoot: 0
2016-05-18 01:33:12,094 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_00, startRequestedTime=1463535192057, startedTime=1463535192094
2016-05-18 01:33:12,094 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,094 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,094 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535191652_0001_1_00 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 2 numSources: 2
2016-05-18 01:33:12,094 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] with distanceFromRoot: 1
2016-05-18 01:33:12,095 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,095 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,095 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,095 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,097 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_02, startRequestedTime=1463535192094, startedTime=1463535192094
2016-05-18 01:33:12,097 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,098 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskId=task_1463535191652_0001_1_00_000000, scheduledTime=1463535192098, launchTime=1463535192098
2016-05-18 01:33:12,098 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,098 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskId=task_1463535191652_0001_1_01_000000, scheduledTime=1463535192098, launchTime=1463535192098
2016-05-18 01:33:12,099 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,103 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:12,103 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,104 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FE62E2013A1A4DE5B2359A2A3196EDC1 in vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,105 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A24AE922279248E69EE6985AE886DB2D in vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 numBipartiteSourceTasks: 2
2016-05-18 01:33:12,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000001, launchTime=1463535192106
2016-05-18 01:33:12,106 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000002, launchTime=1463535192106
2016-05-18 01:33:12,107 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,110 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,111 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_00_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, startTime=1463535192110, containerId=container_1463535191652_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:12,111 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,112 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_01_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,112 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, startTime=1463535192112, containerId=container_1463535191652_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,113 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,113 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,113 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,114 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: FE62E2013A1A4DE5B2359A2A3196EDC1, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,114 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,114 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,116 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,116 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,116 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: A24AE922279248E69EE6985AE886DB2D, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,116 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,118 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191652_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:12,118 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,122 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,124 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,124 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191652_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:33:12,125 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,132 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,136 INFO  input.MRInput (MRInput.java:initialize(440)) - 963FB9B4FA5843BBB36D210BA6B2CFED using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,136 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,136 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,138 INFO  input.MRInput (MRInput.java:initialize(440)) - 963FB9B4FA5843BBB36D210BA6B2CFED using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,139 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,141 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, ordinal: 1
2016-05-18 01:33:12,141 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, mem on start (mb), free: 171, total: 394, max: 672
2016-05-18 01:33:12,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,142 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,142 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[963FB9B4FA5843BBB36D210BA6B2CFED:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F7C5BC8A38EB422FAAA1BF2B645CE780:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,142 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,143 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,143 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,143 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F7C5BC8A38EB422FAAA1BF2B645CE780:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,143 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 963FB9B4FA5843BBB36D210BA6B2CFED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,144 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,146 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,147 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,147 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A24AE922279248E69EE6985AE886DB2D, ordinal: 0
2016-05-18 01:33:12,147 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24AE922279248E69EE6985AE886DB2D, mem on start (mb), free: 171, total: 394, max: 672
2016-05-18 01:33:12,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[963FB9B4FA5843BBB36D210BA6B2CFED:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F7C5BC8A38EB422FAAA1BF2B645CE780:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,149 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F7C5BC8A38EB422FAAA1BF2B645CE780:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 963FB9B4FA5843BBB36D210BA6B2CFED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,151 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,151 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"] streamed: true, id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,152 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']], id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,152 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,154 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,155 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F7C5BC8A38EB422FAAA1BF2B645CE780 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@38c108c8, comparator=cascading.tuple.hadoop.util.TupleComparator@9279f94, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,155 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false
2016-05-18 01:33:12,171 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:33:12,171 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 963FB9B4FA5843BBB36D210BA6B2CFED initialized RecordReader from event
2016-05-18 01:33:12,182 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,182 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,183 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,183 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F7C5BC8A38EB422FAAA1BF2B645CE780: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12,183 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,183 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,186 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,187 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"] streamed: true, id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,187 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']], id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,187 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,189 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:33:12,189 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 963FB9B4FA5843BBB36D210BA6B2CFED initialized RecordReader from event
2016-05-18 01:33:12,190 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F7C5BC8A38EB422FAAA1BF2B645CE780 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@13432534, comparator=cascading.tuple.hadoop.util.TupleComparator@1d3f82cd, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,191 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false
2016-05-18 01:33:12,192 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,192 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, mem on close (mb), free: 144, total: 394, max: 672
2016-05-18 01:33:12,192 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Starting flush of map output
2016-05-18 01:33:12,192 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Span0.length = 5, perItem = 15
2016-05-18 01:33:12,192 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F7C5BC8A38EB422FAAA1BF2B645CE780: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,192 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:12,193 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_00_000000_0_10072_0/file.out
2016-05-18 01:33:12,207 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,208 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,208 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,207 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12,208 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F7C5BC8A38EB422FAAA1BF2B645CE780: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,210 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,212 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:12,214 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:12,215 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,213 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A24AE922279248E69EE6985AE886DB2D, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,218 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:12,218 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,218 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, creationTime=1463535192098, allocationTime=1463535192104, startTime=1463535192110, finishTime=1463535192218, timeTaken=108, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,219 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,218 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:12,219 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,219 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskId=task_1463535191652_0001_1_00_000000, startTime=1463535192110, finishTime=1463535192219, timeTaken=109, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,220 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,220 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,220 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,221 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, vertexId=vertex_1463535191652_0001_1_00, initRequestedTime=1463535192054, initedTime=1463535192093, startRequestedTime=1463535192057, startedTime=1463535192094, finishTime=1463535192220, timeTaken=126, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192110, firstTasksToStart=[ task_1463535191652_0001_1_00_000000 ], lastTaskFinishTime=1463535192218, lastTasksToFinish=[ task_1463535191652_0001_1_00_000000 ], minTaskDuration=108, maxTaskDuration=108, avgTaskDuration=108.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_00_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,222 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,222 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] attempt: attempt_1463535191652_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,223 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000001, stoppedTime=1463535192223, exitStatus=0
2016-05-18 01:33:12,224 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,224 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,225 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,224 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,225 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24AE922279248E69EE6985AE886DB2D, mem on close (mb), free: 141, total: 394, max: 672
2016-05-18 01:33:12,225 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Starting flush of map output
2016-05-18 01:33:12,226 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Span0.length = 5, perItem = 15
2016-05-18 01:33:12,226 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F7C5BC8A38EB422FAAA1BF2B645CE780: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,227 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:12,228 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_01_000000_0_10074_0/file.out
2016-05-18 01:33:12,231 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,231 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,231 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000001, Exiting
2016-05-18 01:33:12,231 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,231 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,236 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12,236 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,236 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000001 completed successfully
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,237 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:12,237 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:12,238 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,239 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:12,239 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,239 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:12,239 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,240 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, creationTime=1463535192099, allocationTime=1463535192104, startTime=1463535192112, finishTime=1463535192240, timeTaken=128, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,240 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,240 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,241 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskId=task_1463535191652_0001_1_01_000000, startTime=1463535192112, finishTime=1463535192240, timeTaken=128, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,241 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,241 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,242 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, vertexId=vertex_1463535191652_0001_1_01, initRequestedTime=1463535192055, initedTime=1463535192089, startRequestedTime=1463535192057, startedTime=1463535192089, finishTime=1463535192241, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192112, firstTasksToStart=[ task_1463535191652_0001_1_01_000000 ], lastTaskFinishTime=1463535192240, lastTasksToFinish=[ task_1463535191652_0001_1_01_000000 ], minTaskDuration=128, maxTaskDuration=128, avgTaskDuration=128.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_01_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,242 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,243 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000002, stoppedTime=1463535192243, exitStatus=0
2016-05-18 01:33:12,243 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] attempt: attempt_1463535191652_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,243 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,243 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,243 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:12,243 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,244 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A24AE922279248E69EE6985AE886DB2D, destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,244 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,244 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,246 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskId=task_1463535191652_0001_1_02_000000, scheduledTime=1463535192246, launchTime=1463535192246
2016-05-18 01:33:12,247 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,248 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,248 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,249 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000002, Exiting
2016-05-18 01:33:12,249 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,249 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,250 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,250 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000002 completed successfully
2016-05-18 01:33:12,251 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000003, launchTime=1463535192250
2016-05-18 01:33:12,252 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,252 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_02_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,253 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, startTime=1463535192252, containerId=container_1463535191652_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,253 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,254 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: F7C5BC8A38EB422FAAA1BF2B645CE780, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=A24AE922279248E69EE6985AE886DB2D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F1CC6B9FD79F49E0BAE2E82A62FDA501, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] groupInputSpecList=[{Group: F7C5BC8A38EB422FAAA1BF2B645CE780 { A24AE922279248E69EE6985AE886DB2D FE62E2013A1A4DE5B2359A2A3196EDC1 } MergedInputDescriptor: org.apache.tez.runtime.library.input.OrderedGroupedMergedKVInput}, ]
2016-05-18 01:33:12,254 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,256 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,257 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535191652_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,258 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:12,262 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,267 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,274 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, ordinal: 2
2016-05-18 01:33:12,276 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, mem on start (mb), free: 134, total: 394, max: 672
2016-05-18 01:33:12,276 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:12,277 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,275 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,278 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,281 INFO  output.MROutput (MROutput.java:initialize(421)) - F1CC6B9FD79F49E0BAE2E82A62FDA501: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:12,281 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,284 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FE62E2013A1A4DE5B2359A2A3196EDC1:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [F1CC6B9FD79F49E0BAE2E82A62FDA501:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [A24AE922279248E69EE6985AE886DB2D:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:12,285 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,285 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:12,285 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:12,286 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FE62E2013A1A4DE5B2359A2A3196EDC1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [F1CC6B9FD79F49E0BAE2E82A62FDA501:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [A24AE922279248E69EE6985AE886DB2D:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:12,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,286 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(278)) - Ignoring FE62E2013A1A4DE5B2359A2A3196EDC1 for start, since it will be controlled via it's Group
2016-05-18 01:33:12,287 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(278)) - Ignoring A24AE922279248E69EE6985AE886DB2D for start, since it will be controlled via it's Group
2016-05-18 01:33:12,287 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(298)) - InputGroup: F7C5BC8A38EB422FAAA1BF2B645CE780 being auto started by the framework. Subsequent instance will not be auto-started
2016-05-18 01:33:12,287 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A24AE922279248E69EE6985AE886DB2D: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,288 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A24AE922279248E69EE6985AE886DB2D: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,290 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A24AE922279248E69EE6985AE886DB2D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,290 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,293 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,293 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FE62E2013A1A4DE5B2359A2A3196EDC1: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,295 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FE62E2013A1A4DE5B2359A2A3196EDC1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,298 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:12,298 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,300 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,301 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FE62E2013A1A4DE5B2359A2A3196EDC1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,303 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A24AE922279248E69EE6985AE886DB2D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,315 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2016-05-18 01:33:12,315 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] streamed: true, id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,315 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"], id: F1CC6B9FD79F49E0BAE2E82A62FDA501
2016-05-18 01:33:12,316 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535191652_0001_1_00_000000_0_10072} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535192315, TimeTaken=1, Rate=0.09 MB/s
2016-05-18 01:33:12,316 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,316 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"]
2016-05-18 01:33:12,316 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedMergedKVInput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B, for 1 inputs
2016-05-18 01:33:12,316 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,317 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FE62E2013A1A4DE5B2359A2A3196EDC1, wasInterrupted=false
2016-05-18 01:33:12,317 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,318 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:12,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_00_000000_0_10072/file.out, len=95, isLocal=true
2016-05-18 01:33:12,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:12,319 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:12,319 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:12,319 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:12,319 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,320 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Shuffle Runner thread complete
2016-05-18 01:33:12,320 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:12,320 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535191652_0001_1_01_000000_0_10074} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535192320, TimeTaken=17, Rate=0.01 MB/s
2016-05-18 01:33:12,321 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,321 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,321 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A24AE922279248E69EE6985AE886DB2D, wasInterrupted=false
2016-05-18 01:33:12,322 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,323 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:12,323 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_01_000000_0_10074/file.out, len=95, isLocal=true
2016-05-18 01:33:12,324 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:12,324 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:12,324 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:12,324 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:12,324 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,325 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A24AE922279248E69EE6985AE886DB2D: Shuffle Runner thread complete
2016-05-18 01:33:12,325 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, all 1 inputs ready in: 00:00:00.008
2016-05-18 01:33:12,326 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A24AE922279248E69EE6985AE886DB2D: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:12,328 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FE62E2013A1A4DE5B2359A2A3196EDC1: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:12,332 INFO  output.MROutput (MROutput.java:close(526)) - F1CC6B9FD79F49E0BAE2E82A62FDA501 closed
2016-05-18 01:33:12,333 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535191652_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:12,334 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,334 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, mem on close (mb), free: 342, total: 390, max: 672
2016-05-18 01:33:12,334 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,334 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FE62E2013A1A4DE5B2359A2A3196EDC1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:12,334 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,334 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A24AE922279248E69EE6985AE886DB2D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:12,334 INFO  output.MROutput (MROutput.java:close(526)) - F1CC6B9FD79F49E0BAE2E82A62FDA501 closed
2016-05-18 01:33:12,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,335 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:12,335 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:12,336 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=10]]
2016-05-18 01:33:12,338 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:12,338 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, creationTime=1463535192247, allocationTime=1463535192248, startTime=1463535192252, finishTime=1463535192338, timeTaken=86, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,338 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,338 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:12,338 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,338 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,339 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,339 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskId=task_1463535191652_0001_1_02_000000, startTime=1463535192252, finishTime=1463535192338, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10
2016-05-18 01:33:12,339 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,339 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,341 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, vertexId=vertex_1463535191652_0001_1_02, initRequestedTime=1463535192057, initedTime=1463535192064, startRequestedTime=1463535192094, startedTime=1463535192094, finishTime=1463535192340, timeTaken=246, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535192252, firstTasksToStart=[ task_1463535191652_0001_1_02_000000 ], lastTaskFinishTime=1463535192338, lastTasksToFinish=[ task_1463535191652_0001_1_02_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_02_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,341 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,341 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000003, stoppedTime=1463535192341, exitStatus=0
2016-05-18 01:33:12,341 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,341 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,341 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,342 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,342 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535191652_0001_1
2016-05-18 01:33:12,342 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535191652_0001_1
2016-05-18 01:33:12,342 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F1CC6B9FD79F49E0BAE2E82A62FDA501 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], outputName: F1CC6B9FD79F49E0BAE2E82A62FDA501
2016-05-18 01:33:12,345 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:12,345 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,349 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,349 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,350 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000003, Exiting
2016-05-18 01:33:12,350 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,350 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,350 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,350 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000003 completed successfully
2016-05-18 01:33:12,353 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F1CC6B9FD79F49E0BAE2E82A62FDA501 of vertex/vertexGroup:F7C5BC8A38EB422FAAA1BF2B645CE780 isVertexGroupOutput:false
2016-05-18 01:33:12,353 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,356 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535191652_0001_1, queueSize=0
2016-05-18 01:33:12,356 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535191652_0001_1, startTime=1463535192054, finishTime=1463535192353, timeTaken=299, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=780, AM_GC_TIME_MILLIS=25, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605576725, Process_Duration=24, Process_End_Time=4390605576749, Read_Duration=13, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:33:12,356 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535191652_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:12,356 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:12 Completed Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12,357 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535191652_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:12,357 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191652_0001_1_post
2016-05-18 01:33:12,358 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,358 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:12,358 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:12,358 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, with id=dag_1463535191652_0001_1
2016-05-18 01:33:12,358 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, with id=dag_1463535191652_0001_1
2016-05-18 01:33:12,363 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001
2016-05-18 01:33:12,365 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:12,365 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:12,366 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:12,366 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:12,366 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:12,366 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:12,367 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:12,367 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:12,367 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:12,367 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:12,368 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:12,368 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:12,368 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:12,368 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:12,372 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001
2016-05-18 01:33:12,373 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit/_temporary
2016-05-18 01:33:12,373 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs]  completed in: 00:00.745
2016-05-18 01:33:12,448 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:12,448 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit
2016-05-18 01:33:12,473 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:12,473 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:12,512 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:12,523 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 01:33:12,524 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:12,524 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:12,524 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:12,549 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:12,562 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars
2016-05-18 01:33:12,667 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] starting
2016-05-18 01:33:12,668 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,668 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"]
2016-05-18 01:33:12,668 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  parallel execution of steps is enabled: false
2016-05-18 01:33:12,668 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  executing total steps: 1
2016-05-18 01:33:12,668 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  allocating management threads: 1
2016-05-18 01:33:12,669 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] starting step: (1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,669 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [**&%&%bar:bar@foo://bl...] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:12,683 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tez session mode enabled: true
2016-05-18 01:33:12,689 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:12,689 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:12,689 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:12,700 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001 doesn't exist and is created
2016-05-18 01:33:12,723 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:12,723 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:12,724 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd
2016-05-18 01:33:12,742 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535192689_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:12,742 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:12,743 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:12,744 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:12,745 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:12,746 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:12,746 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:12,746 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:12,746 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:12,746 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:12,746 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:12,747 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:12,747 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:12,747 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:12,747 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:12,747 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:12,748 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:12,749 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:12,750 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:12,750 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:12,750 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir
2016-05-18 01:33:12,750 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/history.txt.appattempt_1463535192689_0001_000000, maxErrors=10
2016-05-18 01:33:12,751 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:12,751 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535192689_0001_000000, appSubmitTime=1463535192742, launchTime=1463535192742
2016-05-18 01:33:12,754 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:12,761 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50150
2016-05-18 01:33:12,761 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535192689
2016-05-18 01:33:12,763 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535192689_0001_000000, startTime=1463535192763
2016-05-18 01:33:12,763 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:12,823 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:12,823 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:12,823 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001, dagName=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,841 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,841 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:12,842 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535192689_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/dag_1463535192689_0001_1.dot
2016-05-18 01:33:12,843 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/dag_1463535192689_0001_1-tez-dag.pb.txt
2016-05-18 01:33:12,853 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535192689_0001_1
2016-05-18 01:33:12,853 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,854 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12 Running Dag: dag_1463535192689_0001_1
2016-05-18 01:33:12,868 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535192689_0001_1, submitTime=1463535192841
2016-05-18 01:33:12,869 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,869 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,869 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,870 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,870 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,872 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,872 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,872 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,873 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:12,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535192689_0001_1, initTime=1463535192873
2016-05-18 01:33:12,874 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:12,874 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:12,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_STARTED]: dagID=dag_1463535192689_0001_1, startTime=1463535192874
2016-05-18 01:33:12,874 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:12,874 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B243B3E306934EDA844CBCFB2DC403CD : {35601625C6C44CF4B4693503323B1B33={InputName=35601625C6C44CF4B4693503323B1B33}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 35601625C6C44CF4B4693503323B1B33, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,876 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,876 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 35601625C6C44CF4B4693503323B1B33 on vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,879 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 35601625C6C44CF4B4693503323B1B33 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,879 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,879 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,881 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,881 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001, dagName=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,881 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,882 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, vertexId=vertex_1463535192689_0001_1_01, initRequestedTime=1463535192876, initedTime=1463535192881, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,882 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,882 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,882 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,882 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:12,882 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,883 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,883 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 35601625C6C44CF4B4693503323B1B33 on vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,887 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,887 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,887 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,888 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,888 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], numAdditionalOutputs=1
2016-05-18 01:33:12,888 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=29A042B03F7B4091B3E9D134E2EE9553, vertex=vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:12,890 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A5DFDE462F5E409084AA77724F0033C8:29A042B03F7B4091B3E9D134E2EE9553 using oldmapred API
2016-05-18 01:33:12,891 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=29A042B03F7B4091B3E9D134E2EE9553, vertexName=A5DFDE462F5E409084AA77724F0033C8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:12,891 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, vertexId=vertex_1463535192689_0001_1_02, initRequestedTime=1463535192882, initedTime=1463535192888, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,892 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,893 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,893 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {35601625C6C44CF4B4693503323B1B33=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,894 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] parallelism set to 1
2016-05-18 01:33:12,894 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,894 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, vertexId=vertex_1463535192689_0001_1_00, initRequestedTime=1463535192874, initedTime=1463535192894, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,894 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] with distanceFromRoot: 0
2016-05-18 01:33:12,895 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_00, startRequestedTime=1463535192875, startedTime=1463535192895
2016-05-18 01:33:12,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,895 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,895 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535192689_0001_1_00 for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] numStartedSources: 1 numSources: 1
2016-05-18 01:33:12,895 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] with distanceFromRoot: 1
2016-05-18 01:33:12,896 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_01, startRequestedTime=1463535192895, startedTime=1463535192895
2016-05-18 01:33:12,896 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,896 INFO  impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535192689_0001_1_01 for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] numStartedSources: 1 numSources: 1
2016-05-18 01:33:12,896 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] with distanceFromRoot: 2
2016-05-18 01:33:12,897 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_02, startRequestedTime=1463535192896, startedTime=1463535192897
2016-05-18 01:33:12,897 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,901 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: FDC9A2FF27E24C74964FCCD237F4FB29 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,901 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:12,902 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,901 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B243B3E306934EDA844CBCFB2DC403CD in vertex: FDC9A2FF27E24C74964FCCD237F4FB29 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,902 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskId=task_1463535192689_0001_1_00_000000, scheduledTime=1463535192902, launchTime=1463535192902
2016-05-18 01:33:12,903 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,903 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A5DFDE462F5E409084AA77724F0033C8 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,903 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:12,904 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FDC9A2FF27E24C74964FCCD237F4FB29 in vertex: A5DFDE462F5E409084AA77724F0033C8 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,906 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000001, launchTime=1463535192906
2016-05-18 01:33:12,906 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,907 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_00_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,907 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, startTime=1463535192907, containerId=container_1463535192689_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:12,908 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,909 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] submitted tez dag to app master: application_1463535192689_0001, with dag id: dag_1463535192689_0001_1
2016-05-18 01:33:12,909 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,909 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,912 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,912 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: B243B3E306934EDA844CBCFB2DC403CD, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=35601625C6C44CF4B4693503323B1B33, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,912 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,912 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,913 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535192689_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:12,913 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,915 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,916 INFO  input.MRInput (MRInput.java:initialize(440)) - 35601625C6C44CF4B4693503323B1B33 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,917 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,919 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, ordinal: 0
2016-05-18 01:33:12,919 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, mem on start (mb), free: 209, total: 390, max: 672
2016-05-18 01:33:12,919 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,919 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,920 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,922 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,922 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[35601625C6C44CF4B4693503323B1B33:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FDC9A2FF27E24C74964FCCD237F4FB29:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,922 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,923 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,923 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,923 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[35601625C6C44CF4B4693503323B1B33:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FDC9A2FF27E24C74964FCCD237F4FB29:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,923 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,924 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 35601625C6C44CF4B4693503323B1B33 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,924 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,924 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,924 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,925 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,925 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,927 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:12,927 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 35601625C6C44CF4B4693503323B1B33 initialized RecordReader from event
2016-05-18 01:33:12,929 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,929 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,929 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]], id: A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:12,929 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:12,931 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - FDC9A2FF27E24C74964FCCD237F4FB29 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@66d1439c, comparator=cascading.tuple.hadoop.util.TupleComparator@34adeb80, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,931 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for FDC9A2FF27E24C74964FCCD237F4FB29: , UsingHashComparator=false
2016-05-18 01:33:12,955 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,956 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,956 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for FDC9A2FF27E24C74964FCCD237F4FB29: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,956 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - FDC9A2FF27E24C74964FCCD237F4FB29: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12,956 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,957 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,959 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,959 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, mem on close (mb), free: 244, total: 393, max: 672
2016-05-18 01:33:12,959 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - FDC9A2FF27E24C74964FCCD237F4FB29: Starting flush of map output
2016-05-18 01:33:12,959 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - FDC9A2FF27E24C74964FCCD237F4FB29: Span0.length = 5, perItem = 13
2016-05-18 01:33:12,959 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - FDC9A2FF27E24C74964FCCD237F4FB29: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,961 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - FDC9A2FF27E24C74964FCCD237F4FB29: Heap = SpanIterator<0:4> (span=Span[16777216,65]),
2016-05-18 01:33:12,962 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - FDC9A2FF27E24C74964FCCD237F4FB29: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_00_000000_0_10081_0/file.out
2016-05-18 01:33:12,965 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,965 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:12,965 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:12,966 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,967 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:12,967 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, creationTime=1463535192902, allocationTime=1463535192905, startTime=1463535192907, finishTime=1463535192967, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,967 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,967 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,967 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:12,968 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,968 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskId=task_1463535192689_0001_1_00_000000, startTime=1463535192907, finishTime=1463535192967, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,968 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,968 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,968 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,970 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, vertexId=vertex_1463535192689_0001_1_00, initRequestedTime=1463535192874, initedTime=1463535192894, startRequestedTime=1463535192875, startedTime=1463535192895, finishTime=1463535192968, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192907, firstTasksToStart=[ task_1463535192689_0001_1_00_000000 ], lastTaskFinishTime=1463535192967, lastTasksToFinish=[ task_1463535192689_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_00_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,970 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,970 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] attempt: attempt_1463535192689_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,970 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:12,970 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000001, stoppedTime=1463535192970, exitStatus=0
2016-05-18 01:33:12,970 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B243B3E306934EDA844CBCFB2DC403CD, destinationVertexName=FDC9A2FF27E24C74964FCCD237F4FB29. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,971 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,971 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,971 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,972 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,972 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskId=task_1463535192689_0001_1_01_000000, scheduledTime=1463535192972, launchTime=1463535192972
2016-05-18 01:33:12,972 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,975 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000002, launchTime=1463535192975
2016-05-18 01:33:12,977 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000002
2016-05-18 01:33:12,978 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_01_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,978 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, startTime=1463535192978, containerId=container_1463535192689_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:12,978 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,978 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,978 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000001, Exiting
2016-05-18 01:33:12,979 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,979 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,979 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,980 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,980 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000001 completed successfully
2016-05-18 01:33:12,980 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,981 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,981 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,982 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: FDC9A2FF27E24C74964FCCD237F4FB29, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B243B3E306934EDA844CBCFB2DC403CD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5DFDE462F5E409084AA77724F0033C8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,982 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,982 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,984 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535192689_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,984 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:12,985 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,987 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,992 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, ordinal: 1
2016-05-18 01:33:12,992 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, mem on start (mb), free: 239, total: 393, max: 672
2016-05-18 01:33:12,992 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,992 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B243B3E306934EDA844CBCFB2DC403CD:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [A5DFDE462F5E409084AA77724F0033C8:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,993 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,993 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,993 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 316250512, TotalRequestedScaled: 1.58125256E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.45
2016-05-18 01:33:12,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B243B3E306934EDA844CBCFB2DC403CD:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [A5DFDE462F5E409084AA77724F0033C8:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B243B3E306934EDA844CBCFB2DC403CD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,994 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B243B3E306934EDA844CBCFB2DC403CD: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,995 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B243B3E306934EDA844CBCFB2DC403CD: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,995 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B243B3E306934EDA844CBCFB2DC403CD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:12,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,997 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,997 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B243B3E306934EDA844CBCFB2DC403CD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535192689_0001_1_00_000000_0_10081} to DISK_DIRECT, csize=85, dsize=81, EndTime=1463535192999, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 01:33:12,999 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,000 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,000 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B243B3E306934EDA844CBCFB2DC403CD, wasInterrupted=false
2016-05-18 01:33:13,001 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]]
2016-05-18 01:33:13,001 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,002 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:13,002 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] streamed: true, id: A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:13,002 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]], id: 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,002 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_00_000000_0_10081/file.out, len=85, isLocal=true
2016-05-18 01:33:13,003 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 85 bytes from disk
2016-05-18 01:33:13,003 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:13,003 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:13,003 INFO  element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,003 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 64 bytes
2016-05-18 01:33:13,003 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,003 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B243B3E306934EDA844CBCFB2DC403CD: Shuffle Runner thread complete
2016-05-18 01:33:13,004 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - B243B3E306934EDA844CBCFB2DC403CD: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:13,005 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A5DFDE462F5E409084AA77724F0033C8 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@766e32f2, comparator=cascading.tuple.hadoop.util.TupleComparator@3b28c6f6, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:13,005 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A5DFDE462F5E409084AA77724F0033C8: , UsingHashComparator=false
2016-05-18 01:33:13,017 INFO  impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:13,018 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:13,018 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A5DFDE462F5E409084AA77724F0033C8: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:13,018 INFO  impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A5DFDE462F5E409084AA77724F0033C8: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:13,019 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] A3329097CFFB41DF974267568EB44BEF, for 1 inputs
2016-05-18 01:33:13,019 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:13,019 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B243B3E306934EDA844CBCFB2DC403CD: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:13,021 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,022 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, mem on close (mb), free: 136, total: 393, max: 672
2016-05-18 01:33:13,022 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,022 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B243B3E306934EDA844CBCFB2DC403CD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:13,022 INFO  impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A5DFDE462F5E409084AA77724F0033C8: Starting flush of map output
2016-05-18 01:33:13,023 INFO  impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A5DFDE462F5E409084AA77724F0033C8: Span0.length = 5, perItem = 13
2016-05-18 01:33:13,023 INFO  impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A5DFDE462F5E409084AA77724F0033C8: done sorting span=0, length=5, time=0
2016-05-18 01:33:13,023 INFO  impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A5DFDE462F5E409084AA77724F0033C8: Heap = SpanIterator<0:4> (span=Span[16777216,65]),
2016-05-18 01:33:13,024 INFO  impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A5DFDE462F5E409084AA77724F0033C8: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_01_000000_0_10084_0/file.out
2016-05-18 01:33:13,027 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,028 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:13,028 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:13,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_01_000000_0: Counters: 40 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0]]
2016-05-18 01:33:13,029 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:13,029 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,030 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:13,030 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, creationTime=1463535192972, allocationTime=1463535192975, startTime=1463535192978, finishTime=1463535193030, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,030 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,030 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,031 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,031 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskId=task_1463535192689_0001_1_01_000000, startTime=1463535192978, finishTime=1463535193031, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_01_000000_0, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0
2016-05-18 01:33:13,032 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,032 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:13,033 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, vertexId=vertex_1463535192689_0001_1_01, initRequestedTime=1463535192876, initedTime=1463535192881, startRequestedTime=1463535192895, startedTime=1463535192895, finishTime=1463535193032, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, vertexStats=firstTaskStartTime=1463535192978, firstTasksToStart=[ task_1463535192689_0001_1_01_000000 ], lastTaskFinishTime=1463535193030, lastTasksToFinish=[ task_1463535192689_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_01_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,033 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,033 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000002, stoppedTime=1463535193033, exitStatus=0
2016-05-18 01:33:13,034 INFO  impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] attempt: attempt_1463535192689_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:13,034 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A5DFDE462F5E409084AA77724F0033C8
2016-05-18 01:33:13,034 INFO  impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, destinationVertexName=A5DFDE462F5E409084AA77724F0033C8. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:13,034 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,034 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:13,035 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,035 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,035 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskId=task_1463535192689_0001_1_02_000000, scheduledTime=1463535193035, launchTime=1463535193035
2016-05-18 01:33:13,035 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,037 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000003, launchTime=1463535193036
2016-05-18 01:33:13,037 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,038 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_02_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:13,038 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, startTime=1463535193038, containerId=container_1463535192689_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:13,038 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,039 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,039 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,039 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:13,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: A5DFDE462F5E409084AA77724F0033C8, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=29A042B03F7B4091B3E9D134E2EE9553, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:13,039 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:13,040 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,040 INFO  impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535192689_0001_1_02_000000_0 1 events [0,1) total 1 vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,041 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:13,053 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,053 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000002: 22 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,054 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000002, Exiting
2016-05-18 01:33:13,054 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,054 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,054 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,054 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000002 completed successfully
2016-05-18 01:33:13,055 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:13,059 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, ordinal: 2
2016-05-18 01:33:13,060 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, mem on start (mb), free: 346, total: 394, max: 672
2016-05-18 01:33:13,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:13,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:13,061 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9,  for final merged output=0, using factor: 0.0
2016-05-18 01:33:13,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:13,066 INFO  output.MROutput (MROutput.java:initialize(421)) - 29A042B03F7B4091B3E9D134E2EE9553: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:13,066 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:13,066 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[29A042B03F7B4091B3E9D134E2EE9553:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [FDC9A2FF27E24C74964FCCD237F4FB29:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:13,066 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:13,066 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:13,067 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:13,067 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[29A042B03F7B4091B3E9D134E2EE9553:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [FDC9A2FF27E24C74964FCCD237F4FB29:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:13,067 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:13,068 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FDC9A2FF27E24C74964FCCD237F4FB29 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:13,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:13,069 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:13,071 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FDC9A2FF27E24C74964FCCD237F4FB29: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:13,071 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FDC9A2FF27E24C74964FCCD237F4FB29: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:13,072 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FDC9A2FF27E24C74964FCCD237F4FB29 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:13,073 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,074 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:13,074 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,075 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FDC9A2FF27E24C74964FCCD237F4FB29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:13,076 INFO  orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535192689_0001_1_01_000000_0_10084} to DISK_DIRECT, csize=85, dsize=81, EndTime=1463535193076, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 01:33:13,076 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,076 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,077 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FDC9A2FF27E24C74964FCCD237F4FB29, wasInterrupted=false
2016-05-18 01:33:13,077 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,078 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]]
2016-05-18 01:33:13,078 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] streamed: true, id: 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,078 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:13,078 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"], id: 29A042B03F7B4091B3E9D134E2EE9553
2016-05-18 01:33:13,079 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"]
2016-05-18 01:33:13,079 INFO  element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(**&%&%bar:bar@foo://blah/	(*(**^**&%&%^@#@&&() :::: ///\\ 	 illegal chars in it)[by:[{1}:0]] 9A60F89A455B4AE5A210FEEE435340A2, for 1 inputs
2016-05-18 01:33:13,079 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_01_000000_0_10084/file.out, len=85, isLocal=true
2016-05-18 01:33:13,079 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 85 bytes from disk
2016-05-18 01:33:13,079 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:13,079 INFO  impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:13,080 INFO  impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 64 bytes
2016-05-18 01:33:13,080 INFO  orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,080 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FDC9A2FF27E24C74964FCCD237F4FB29: Shuffle Runner thread complete
2016-05-18 01:33:13,080 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:33:13,080 INFO  input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FDC9A2FF27E24C74964FCCD237F4FB29: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:13,083 INFO  output.MROutput (MROutput.java:close(526)) - 29A042B03F7B4091B3E9D134E2EE9553 closed
2016-05-18 01:33:13,083 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535192689_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:13,084 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,084 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, mem on close (mb), free: 343, total: 394, max: 672
2016-05-18 01:33:13,084 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,084 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FDC9A2FF27E24C74964FCCD237F4FB29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:13,084 INFO  output.MROutput (MROutput.java:close(526)) - 29A042B03F7B4091B3E9D134E2EE9553 closed
2016-05-18 01:33:13,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,085 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:13,085 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:13,086 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:13,087 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:13,087 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, creationTime=1463535193035, allocationTime=1463535193036, startTime=1463535193038, finishTime=1463535193087, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,087 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,087 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,088 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:13,088 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,088 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,088 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskId=task_1463535192689_0001_1_02_000000, startTime=1463535193038, finishTime=1463535193087, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:13,089 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,089 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, vertexId=vertex_1463535192689_0001_1_02, initRequestedTime=1463535192882, initedTime=1463535192888, startRequestedTime=1463535192896, startedTime=1463535192897, finishTime=1463535193089, timeTaken=192, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535193038, firstTasksToStart=[ task_1463535192689_0001_1_02_000000 ], lastTaskFinishTime=1463535193087, lastTasksToFinish=[ task_1463535192689_0001_1_02_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_02_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,090 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,090 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000003, stoppedTime=1463535193090, exitStatus=0
2016-05-18 01:33:13,091 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:13,091 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,091 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:13,091 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:13,091 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535192689_0001_1
2016-05-18 01:33:13,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535192689_0001_1
2016-05-18 01:33:13,092 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 29A042B03F7B4091B3E9D134E2EE9553 for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], outputName: 29A042B03F7B4091B3E9D134E2EE9553
2016-05-18 01:33:13,093 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:13,093 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,099 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,099 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,099 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000003, Exiting
2016-05-18 01:33:13,099 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,099 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,100 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,100 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000003 completed successfully
2016-05-18 01:33:13,101 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:29A042B03F7B4091B3E9D134E2EE9553 of vertex/vertexGroup:A5DFDE462F5E409084AA77724F0033C8 isVertexGroupOutput:false
2016-05-18 01:33:13,101 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,104 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535192689_0001_1, queueSize=0
2016-05-18 01:33:13,105 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535192689_0001_1, startTime=1463535192874, finishTime=1463535193101, timeTaken=227, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=450, AM_GC_TIME_MILLIS=22, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=130, OUTPUT_BYTES_WITH_OVERHEAD=162, OUTPUT_BYTES_PHYSICAL=170, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=170, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=170, SHUFFLE_BYTES_DECOMPRESSED=162, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=170, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605579056, Process_Duration=8, Process_End_Time=4390605579064, Read_Duration=0, Tuples_Read=15, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:13,105 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535192689_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:13,105 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:13 Completed Dag: dag_1463535192689_0001_1
2016-05-18 01:33:13,105 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535192689_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:13,105 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535192689_0001_1_post
2016-05-18 01:33:13,105 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,105 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:13,106 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:13,106 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars, with id=dag_1463535192689_0001_1
2016-05-18 01:33:13,106 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/	(*(**^/(1/1) ...sInTempFiles/illegalchars, with id=dag_1463535192689_0001_1
2016-05-18 01:33:13,114 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001
2016-05-18 01:33:13,116 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:13,116 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:13,116 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:13,116 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:13,116 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:13,116 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:13,118 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:13,118 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:13,118 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:13,119 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,119 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:13,120 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:13,120 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,121 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:13,125 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001
2016-05-18 01:33:13,126 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars/_temporary
2016-05-18 01:33:13,126 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...]  completed in: 00:00.459
2016-05-18 01:33:13,164 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars
2016-05-18 01:33:13,190 INFO  util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:13,190 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:13,222 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-18 01:33:13,227 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:33:13,227 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:13,228 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:13,228 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:13,243 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:13,243 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"]
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  parallel execution of steps is enabled: false
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  executing total steps: 1
2016-05-18 01:33:13,252 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  allocating management threads: 1
2016-05-18 01:33:13,254 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,254 WARN  flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:13,268 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:13,272 INFO  client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:13,273 INFO  client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:13,273 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:13,282 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001 doesn't exist and is created
2016-05-18 01:33:13,301 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:13,301 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:13,302 INFO  client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd
2016-05-18 01:33:13,317 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535193273_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:13,317 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:13,317 INFO  shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:13,319 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:13,319 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:13,320 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:13,320 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:13,321 INFO  app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:13,322 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:13,322 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:13,323 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:13,323 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:13,323 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:13,323 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:13,323 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:13,324 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:13,324 INFO  common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:13,324 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:13,324 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:13,324 INFO  launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:13,326 INFO  common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:13,327 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:13,328 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:13,328 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir
2016-05-18 01:33:13,328 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/history.txt.appattempt_1463535193273_0001_000000, maxErrors=10
2016-05-18 01:33:13,329 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:13,329 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535193273_0001_000000, appSubmitTime=1463535193316, launchTime=1463535193316
2016-05-18 01:33:13,335 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:13,338 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46154
2016-05-18 01:33:13,339 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535193273
2016-05-18 01:33:13,341 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535193273_0001_000000, startTime=1463535193341
2016-05-18 01:33:13,341 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:13,401 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:13,401 INFO  client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:13,402 INFO  client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001, dagName=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,419 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,419 INFO  app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:13,420 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535193273_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/dag_1463535193273_0001_1.dot
2016-05-18 01:33:13,421 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/dag_1463535193273_0001_1-tez-dag.pb.txt
2016-05-18 01:33:13,425 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535193273_0001_1
2016-05-18 01:33:13,425 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,426 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13 Running Dag: dag_1463535193273_0001_1
2016-05-18 01:33:13,441 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535193273_0001_1, submitTime=1463535193419
2016-05-18 01:33:13,442 INFO  impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001/tez-conf.pb" } size: 40938 timestamp: 1463535193000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:13,442 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,443 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,443 INFO  impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:13,443 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:13,444 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535193273_0001_1, initTime=1463535193444
2016-05-18 01:33:13,444 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:13,445 INFO  impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:13,445 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_STARTED]: dagID=dag_1463535193273_0001_1, startTime=1463535193445
2016-05-18 01:33:13,445 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:13,445 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D5FB588337814B8BA6086D4A928CBB08 : {87E2105144FB4566BF81561E0E4E1D18={InputName=87E2105144FB4566BF81561E0E4E1D18}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:13,445 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 87E2105144FB4566BF81561E0E4E1D18, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:13,445 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:13,447 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 87E2105144FB4566BF81561E0E4E1D18 on vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,447 INFO  client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001, dagName=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,450 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 87E2105144FB4566BF81561E0E4E1D18 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:13,452 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:13,452 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large.  Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:13,453 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:13,453 INFO  grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:13,453 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 87E2105144FB4566BF81561E0E4E1D18 on vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,456 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:13,456 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {87E2105144FB4566BF81561E0E4E1D18=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:13,456 INFO  impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] parallelism set to 1
2016-05-18 01:33:13,456 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:13,457 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], numAdditionalOutputs=1
2016-05-18 01:33:13,457 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C52A942D71F2402D9AE4485BCA5014B8, vertex=vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:13,458 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D5FB588337814B8BA6086D4A928CBB08:C52A942D71F2402D9AE4485BCA5014B8 using oldmapred API
2016-05-18 01:33:13,458 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C52A942D71F2402D9AE4485BCA5014B8, vertexName=D5FB588337814B8BA6086D4A928CBB08, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:13,459 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, vertexId=vertex_1463535193273_0001_1_00, initRequestedTime=1463535193445, initedTime=1463535193457, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:13,460 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] with distanceFromRoot: 0
2016-05-18 01:33:13,460 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:13,460 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535193273_0001_1_00, startRequestedTime=1463535193447, startedTime=1463535193460
2016-05-18 01:33:13,461 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535193273_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:13,461 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:13,461 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D5FB588337814B8BA6086D4A928CBB08
2016-05-18 01:33:13,461 INFO  impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,462 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_STARTED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskId=task_1463535193273_0001_1_00_000000, scheduledTime=1463535193462, launchTime=1463535193462
2016-05-18 01:33:13,462 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,463 INFO  node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:13,465 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535193273_0001_00_000001, launchTime=1463535193464
2016-05-18 01:33:13,465 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,466 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535193273_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,466 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535193273_0001_1_00_000000_0] started. Is using containerId: [container_1463535193273_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:13,467 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, startTime=1463535193466, containerId=container_1463535193273_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:13,467 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535193273_0001, with dag id: dag_1463535193273_0001_1
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,467 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,467 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,467 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:13,468 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, VertexName: D5FB588337814B8BA6086D4A928CBB08, VertexParallelism: 1, TaskAttemptID:attempt_1463535193273_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87E2105144FB4566BF81561E0E4E1D18, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C52A942D71F2402D9AE4485BCA5014B8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:13,468 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:13,469 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,470 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535193273_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:13,470 INFO  task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535193273_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:13,473 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:13,475 INFO  input.MRInput (MRInput.java:initialize(440)) - 87E2105144FB4566BF81561E0E4E1D18 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:13,475 INFO  input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,480 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, ordinal: 0
2016-05-18 01:33:13,481 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, mem on start (mb), free: 266, total: 394, max: 672
2016-05-18 01:33:13,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:13,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:13,481 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:13,485 INFO  output.MROutput (MROutput.java:initialize(421)) - C52A942D71F2402D9AE4485BCA5014B8: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:13,486 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:13,486 INFO  resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[87E2105144FB4566BF81561E0E4E1D18:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [C52A942D71F2402D9AE4485BCA5014B8:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:13,486 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:13,486 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:13,488 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:13,488 INFO  resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[87E2105144FB4566BF81561E0E4E1D18:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [C52A942D71F2402D9AE4485BCA5014B8:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:13,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:13,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 87E2105144FB4566BF81561E0E4E1D18 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:13,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:13,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:13,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,489 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:13,492 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,494 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:13,495 INFO  input.MRInput (MRInput.java:initFromEventInternal(653)) - 87E2105144FB4566BF81561E0E4E1D18 initialized RecordReader from event
2016-05-18 01:33:13,495 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,496 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,496 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"], id: C52A942D71F2402D9AE4485BCA5014B8
2016-05-18 01:33:13,496 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"]
2016-05-18 01:33:13,497 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,497 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:13,499 INFO  output.MROutput (MROutput.java:close(526)) - C52A942D71F2402D9AE4485BCA5014B8 closed
2016-05-18 01:33:13,500 INFO  impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535193273_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:13,500 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,500 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, mem on close (mb), free: 264, total: 394, max: 672
2016-05-18 01:33:13,501 INFO  output.MROutput (MROutput.java:close(526)) - C52A942D71F2402D9AE4485BCA5014B8 closed
2016-05-18 01:33:13,501 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,502 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:13,502 INFO  task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535193273_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:13,502 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535193273_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-18 01:33:13,503 INFO  task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535193273_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}  
2016-05-18 01:33:13,503 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, creationTime=1463535193462, allocationTime=1463535193464, startTime=1463535193466, finishTime=1463535193503, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,503 INFO  task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,504 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: 
2016-05-18 01:33:13,504 INFO  impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,504 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskId=task_1463535193273_0001_1_00_000000, startTime=1463535193466, finishTime=1463535193504, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463535193273_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:33:13,504 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,505 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,505 INFO  impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, vertexId=vertex_1463535193273_0001_1_00, initRequestedTime=1463535193445, initedTime=1463535193457, startRequestedTime=1463535193447, startedTime=1463535193460, finishTime=1463535193505, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535193466, firstTasksToStart=[ task_1463535193273_0001_1_00_000000 ], lastTaskFinishTime=1463535193503, lastTasksToFinish=[ task_1463535193273_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535193273_0001_1_00_000000 ], longestDurationTasks=[ task_1463535193273_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,506 INFO  impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535193273_0001_00_000001, stoppedTime=1463535193506, exitStatus=0
2016-05-18 01:33:13,506 INFO  impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:13,506 INFO  impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,507 INFO  impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535193273_0001_1
2016-05-18 01:33:13,507 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535193273_0001_1
2016-05-18 01:33:13,507 INFO  impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C52A942D71F2402D9AE4485BCA5014B8 for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], outputName: C52A942D71F2402D9AE4485BCA5014B8
2016-05-18 01:33:13,510 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:13,510 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535193273_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,515 INFO  app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535193273_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,515 INFO  task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535193273_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,516 INFO  task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535193273_0001_00_000001, Exiting
2016-05-18 01:33:13,516 INFO  task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,516 INFO  task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,516 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535193273_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,517 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535193273_0001_00_000001 completed successfully
2016-05-18 01:33:13,519 INFO  impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C52A942D71F2402D9AE4485BCA5014B8 of vertex/vertexGroup:D5FB588337814B8BA6086D4A928CBB08 isVertexGroupOutput:false
2016-05-18 01:33:13,519 INFO  impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,520 INFO  recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535193273_0001_1, queueSize=0
2016-05-18 01:33:13,520 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535193273_0001_1, startTime=1463535193445, finishTime=1463535193519, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:33:13,520 INFO  impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535193273_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:13,521 INFO  impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:13 Completed Dag: dag_1463535193273_0001_1
2016-05-18 01:33:13,521 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535193273_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:13,521 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535193273_0001_1_post
2016-05-18 01:33:13,521 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,521 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:13,521 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:13,521 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, with id=dag_1463535193273_0001_1
2016-05-18 01:33:13,521 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, with id=dag_1463535193273_0001_1
2016-05-18 01:33:13,528 INFO  client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001
2016-05-18 01:33:13,529 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:13,529 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:13,530 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:13,530 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:13,530 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:13,530 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:13,531 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:13,531 INFO  app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:13,531 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:13,532 INFO  recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,532 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:13,533 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:13,533 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,534 INFO  rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:13,536 INFO  app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001
2016-05-18 01:33:13,537 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic/_temporary
2016-05-18 01:33:13,537 INFO  flow.Flow (BaseFlow.java:logInfo(1504)) - [test]  completed in: 00:00.284
2016-05-18 01:33:13,572 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic

Standard error

2016-05-18 01:32:57 Running Dag: dag_1463535176640_0001_1
2016-05-18 01:32:57 Starting to run new task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58 Completed Dag: dag_1463535176640_0001_1
2016-05-18 01:32:58 Running Dag: dag_1463535178521_0001_1
2016-05-18 01:32:58 Starting to run new task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58 Completed Dag: dag_1463535178521_0001_1
2016-05-18 01:32:59 Running Dag: dag_1463535179407_0001_1
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_00_000000_0
rhs-pre: ['num', 'char']
rhs-pre: ['1', 'a']
rhs-post: ['num2']
rhs-post: ['1']
rhs-pre: ['2', 'b']
rhs-post: ['2']
rhs-pre: ['3', 'c']
rhs-post: ['3']
rhs-pre: ['4', 'd']
rhs-post: ['4']
rhs-pre: ['5', 'e']
rhs-post: ['5']
rhs-pre: tuples count: 5
rhs-post: tuples count: 5
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00 Starting to run new task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00 Completed Dag: dag_1463535179407_0001_1
2016-05-18 01:33:01 Running Dag: dag_1463535180903_0001_1
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01 Completed Dag: dag_1463535180903_0001_1
2016-05-18 01:33:02 Running Dag: dag_1463535181911_0001_1
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02 Completed Dag: dag_1463535181911_0001_1
cascading.flow.FlowException: local step failed: (1/1) .../testOOMEPreGroup/oomepre
	at cascading.flow.planner.FlowStepJob.blockOnJob(FlowStepJob.java:293)
	at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:184)
	at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:146)
	at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:48)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:02 Running Dag: dag_1463535182367_0001_1
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02 Completed running task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02 Completed Dag: dag_1463535182367_0001_1
2016-05-18 01:33:03 Running Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03 Completed Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03 Running Dag: dag_1463535183398_0001_1
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03 Completed Dag: dag_1463535183398_0001_1
2016-05-18 01:33:04 Running Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04 Completed Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04 Running Dag: dag_1463535184511_0001_1
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04 Completed Dag: dag_1463535184511_0001_1
cascading.flow.FlowException: local step failed: (1/1) ...estOOMEPostGroup/oomepost
	at cascading.flow.planner.FlowStepJob.blockOnJob(FlowStepJob.java:293)
	at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:184)
	at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:146)
	at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:48)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:05 Running Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05 Completed Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05 Running Dag: dag_1463535185729_0001_1
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05 Completed Dag: dag_1463535185729_0001_1
2016-05-18 01:33:06 Running Dag: dag_1463535186170_0001_1
2016-05-18 01:33:06 Starting to run new task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06 Completed running task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06 Completed Dag: dag_1463535186170_0001_1
2016-05-18 01:33:09 Running Dag: dag_1463535189750_0001_1
2016-05-18 01:33:09 Starting to run new task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10 Starting to run new task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10 Completed Dag: dag_1463535189750_0001_1
2016-05-18 01:33:11 Running Dag: dag_1463535191167_0001_1
2016-05-18 01:33:11 Starting to run new task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11 Completed running task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11 Completed Dag: dag_1463535191167_0001_1
2016-05-18 01:33:12 Running Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12 Completed Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12 Running Dag: dag_1463535192689_0001_1
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13 Completed Dag: dag_1463535192689_0001_1
2016-05-18 01:33:13 Running Dag: dag_1463535193273_0001_1
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13 Completed Dag: dag_1463535193273_0001_1