2016-05-18 01:32:54,383 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 01:32:54,417 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.RegressionPipesPlatformTest, with platform: hadoop2-tez
2016-05-18 01:32:54,454 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 01:32:55,056 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 01:32:55,386 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 01:32:55,663 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:55,665 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:55,757 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 8440BE39258F4F0AB26054026672DE42
2016-05-18 01:32:56,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.102
2016-05-18 01:32:56,021 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.116
2016-05-18 01:32:56,022 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:56,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:56,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:56,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:32:56,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor
2016-05-18 01:32:56,431 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 01:32:56,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:32:56,435 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:56,435 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"]
2016-05-18 01:32:56,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:32:56,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:32:56,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:32:56,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...regression/complexlogicor
2016-05-18 01:32:56,438 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:56,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:32:56,634 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:56,640 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:56,641 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:56,706 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001 doesn't exist and is created
2016-05-18 01:32:56,800 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:56,801 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:56,802 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd
2016-05-18 01:32:56,868 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535176640_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:56,901 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:56,901 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 01:32:56,901 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 01:32:56,902 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:56,902 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:56,915 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:56,916 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:56,941 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:56,969 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:56,970 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:56,971 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:56,972 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:56,973 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:56,973 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:56,974 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:56,975 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:56,976 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:56,976 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:56,987 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:56,990 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:57,002 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 01:32:57,002 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 01:32:57,002 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:57,015 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:57,018 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:57,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:57,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir
2016-05-18 01:32:57,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/history.txt.appattempt_1463535176640_0001_000000, maxErrors=10
2016-05-18 01:32:57,022 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:57,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535176640_0001_000000, appSubmitTime=1463535176846, launchTime=1463535176866
2016-05-18 01:32:57,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:57,092 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40193
2016-05-18 01:32:57,094 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535176640
2016-05-18 01:32:57,103 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 01:32:57,103 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 01:32:57,103 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:57,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535176640_0001_000000, startTime=1463535177113
2016-05-18 01:32:57,114 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:57,203 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:57,220 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:57,220 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001, dagName=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,432 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,440 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:57,494 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535176640_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/dag_1463535176640_0001_1.dot
2016-05-18 01:32:57,498 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001_wd/localmode-log-dir/dag_1463535176640_0001_1-tez-dag.pb.txt
2016-05-18 01:32:57,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535176640_0001_1
2016-05-18 01:32:57,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:57,534 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57 Running Dag: dag_1463535176640_0001_1
2016-05-18 01:32:57,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535176640_0001_1, submitTime=1463535177432
2016-05-18 01:32:57,614 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001/tez-conf.pb" } size: 40938 timestamp: 1463535176000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:57,614 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,615 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,615 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:57,617 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:57,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535176640_0001_1, initTime=1463535177619
2016-05-18 01:32:57,620 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:57,624 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:57,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_STARTED]: dagID=dag_1463535176640_0001_1, startTime=1463535177627
2016-05-18 01:32:57,628 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:57,628 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E8FEF806C7D54B23A902AB6C1336327C : {6F1E22DC3F75464282FD5A0697F8F54A={InputName=6F1E22DC3F75464282FD5A0697F8F54A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:57,630 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 6F1E22DC3F75464282FD5A0697F8F54A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:57,630 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,639 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,640 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,641 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001, dagName=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor
2016-05-18 01:32:57,643 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,652 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 6F1E22DC3F75464282FD5A0697F8F54A on vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535176640_0001, with dag id: dag_1463535176640_0001_1
2016-05-18 01:32:57,686 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:57,686 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 6F1E22DC3F75464282FD5A0697F8F54A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:57,719 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:57,719 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:32:57,720 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:57,721 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:57,737 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 6F1E22DC3F75464282FD5A0697F8F54A on vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,745 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:57,746 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {6F1E22DC3F75464282FD5A0697F8F54A=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:57,759 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] parallelism set to 1
2016-05-18 01:32:57,759 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:57,794 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], numAdditionalOutputs=1
2016-05-18 01:32:57,794 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=87340D55D24749218144C739866F0086, vertex=vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:57,803 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E8FEF806C7D54B23A902AB6C1336327C:87340D55D24749218144C739866F0086 using oldmapred API
2016-05-18 01:32:57,804 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=87340D55D24749218144C739866F0086, vertexName=E8FEF806C7D54B23A902AB6C1336327C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:57,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, vertexId=vertex_1463535176640_0001_1_00, initRequestedTime=1463535177628, initedTime=1463535177794, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:57,815 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] with distanceFromRoot: 0
2016-05-18 01:32:57,815 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:57,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535176640_0001_1_00, startRequestedTime=1463535177686, startedTime=1463535177815
2016-05-18 01:32:57,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535176640_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:57,819 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:57,820 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E8FEF806C7D54B23A902AB6C1336327C
2016-05-18 01:32:57,820 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_STARTED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskId=task_1463535176640_0001_1_00_000000, scheduledTime=1463535177824, launchTime=1463535177824
2016-05-18 01:32:57,849 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:57,891 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:57,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535176640_0001_00_000001, launchTime=1463535177907
2016-05-18 01:32:57,912 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535176640_0001_00_000001
2016-05-18 01:32:57,916 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535176640_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57 Starting to run new task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,917 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,918 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:57,919 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:57,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535176640_0001_1_00_000000_0] started. Is using containerId: [container_1463535176640_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:57,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, startTime=1463535177916, containerId=container_1463535176640_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:57,921 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 01:32:57,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, VertexName: E8FEF806C7D54B23A902AB6C1336327C, VertexParallelism: 1, TaskAttemptID:attempt_1463535176640_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6F1E22DC3F75464282FD5A0697F8F54A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=87340D55D24749218144C739866F0086, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:32:57,935 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:57,939 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:57,948 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535176640_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:57,950 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535176640_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:57,973 INFO input.MRInput (MRInput.java:initialize(440)) - 6F1E22DC3F75464282FD5A0697F8F54A using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:57,974 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:57,975 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:57,991 INFO output.MROutput (MROutput.java:initialize(421)) - 87340D55D24749218144C739866F0086: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:32:57,996 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, ordinal: 0
2016-05-18 01:32:57,997 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, mem on start (mb), free: 243, total: 281, max: 672
2016-05-18 01:32:57,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:57,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:57,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:57,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:57,998 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6F1E22DC3F75464282FD5A0697F8F54A:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [87340D55D24749218144C739866F0086:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:32:58,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:58,007 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:58,007 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:32:58,007 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6F1E22DC3F75464282FD5A0697F8F54A:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [87340D55D24749218144C739866F0086:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:32:58,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:58,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:58,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 6F1E22DC3F75464282FD5A0697F8F54A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:58,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:58,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:58,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:58,012 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,028 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:32:58,029 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 6F1E22DC3F75464282FD5A0697F8F54A initialized RecordReader from event
2016-05-18 01:32:58,047 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:58,062 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 6F1E22DC3F75464282FD5A0697F8F54A
2016-05-18 01:32:58,063 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"], id: 87340D55D24749218144C739866F0086
2016-05-18 01:32:58,063 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor"]
2016-05-18 01:32:58,067 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:32:58,069 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:32:58,078 INFO output.MROutput (MROutput.java:close(526)) - 87340D55D24749218144C739866F0086 closed
2016-05-18 01:32:58,082 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535176640_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:32:58,084 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,084 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8FEF806C7D54B23A902AB6C1336327C, mem on close (mb), free: 237, total: 281, max: 672
2016-05-18 01:32:58,085 INFO output.MROutput (MROutput.java:close(526)) - 87340D55D24749218144C739866F0086 closed
2016-05-18 01:32:58,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,087 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:32:58,088 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535176640_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:32:58,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535176640_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4][cascading.flow.SliceCounters Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=4]]
2016-05-18 01:32:58,096 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535176640_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskAttemptId=attempt_1463535176640_0001_1_00_000000_0, creationTime=1463535177842, allocationTime=1463535177901, startTime=1463535177916, finishTime=1463535178095, timeTaken=179, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:32:58,102 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535176640_0001_1_00_000000_0
2016-05-18 01:32:58,104 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,105 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:32:58,105 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,107 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:TASK_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, taskId=task_1463535176640_0001_1_00_000000, startTime=1463535177916, finishTime=1463535178103, timeTaken=187, status=SUCCEEDED, successfulAttemptID=attempt_1463535176640_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4
2016-05-18 01:32:58,116 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:32:58,116 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C]
2016-05-18 01:32:58,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:VERTEX_FINISHED]: vertexName=E8FEF806C7D54B23A902AB6C1336327C, vertexId=vertex_1463535176640_0001_1_00, initRequestedTime=1463535177628, initedTime=1463535177794, startRequestedTime=1463535177686, startedTime=1463535177815, finishTime=1463535178117, timeTaken=302, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4, vertexStats=firstTaskStartTime=1463535177916, firstTasksToStart=[ task_1463535176640_0001_1_00_000000 ], lastTaskFinishTime=1463535178095, lastTasksToFinish=[ task_1463535176640_0001_1_00_000000 ], minTaskDuration=179, maxTaskDuration=179, avgTaskDuration=179.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535176640_0001_1_00_000000 ], longestDurationTasks=[ task_1463535176640_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:32:58,134 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:32:58,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535176640_0001_00_000001, stoppedTime=1463535178136, exitStatus=0
2016-05-18 01:32:58,137 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:32:58,138 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,138 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535176640_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:32:58,138 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535176640_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:32:58,139 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535176640_0001_00_000001, Exiting
2016-05-18 01:32:58,139 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,139 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535176640_0001_00_000001
2016-05-18 01:32:58,140 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535176640_0001_1
2016-05-18 01:32:58,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535176640_0001_1
2016-05-18 01:32:58,142 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535176640_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:32:58,143 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535176640_0001_00_000001 completed successfully
2016-05-18 01:32:58,143 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:32:58,143 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 87340D55D24749218144C739866F0086 for vertex: vertex_1463535176640_0001_1_00 [E8FEF806C7D54B23A902AB6C1336327C], outputName: 87340D55D24749218144C739866F0086
2016-05-18 01:32:58,144 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463535176640_0001_00_000001
2016-05-18 01:32:58,161 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:87340D55D24749218144C739866F0086 of vertex/vertexGroup:E8FEF806C7D54B23A902AB6C1336327C isVertexGroupOutput:false
2016-05-18 01:32:58,161 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,169 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535176640_0001_1, queueSize=0
2016-05-18 01:32:58,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535176640_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535176640_0001_1, startTime=1463535177624, finishTime=1463535178161, timeTaken=537, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1280, AM_GC_TIME_MILLIS=24, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, cascading.flow.SliceCounters, Process_Begin_Time=1463535178070, Process_Duration=14, Process_End_Time=1463535178084, Read_Duration=1, Tuples_Read=13, Tuples_Written=4, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=4
2016-05-18 01:32:58,174 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535176640_0001_1 finished with state: SUCCEEDED
2016-05-18 01:32:58,174 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535176640_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:32:58 Completed Dag: dag_1463535176640_0001_1
2016-05-18 01:32:58,176 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535176640_0001_1, dagState=SUCCEEDED
2016-05-18 01:32:58,179 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535176640_0001_1_post
2016-05-18 01:32:58,179 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,181 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:32:58,181 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:32:58,186 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, with id=dag_1463535176640_0001_1
2016-05-18 01:32:58,188 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...regression/complexlogicor, applicationId=application_1463535176640_0001
2016-05-18 01:32:58,191 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD] test/(1/1) ...regression/complexlogicor, with id=dag_1463535176640_0001_1
2016-05-18 01:32:58,197 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:32:58,197 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:32:58,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:32:58,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:32:58,198 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:32:58,201 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:32:58,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:32:58,203 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:32:58,204 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:32:58,204 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,206 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:32:58,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:32:58,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,208 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:32:58,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/0C2588BEFFFF45E8B25BB0761BD0FE49/BD713A31BA374FFE8BD0B85385F44BCD/.tez/application_1463535176640_0001
2016-05-18 01:32:58,217 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor/_temporary
2016-05-18 01:32:58,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:01.781
2016-05-18 01:32:58,272 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicOr/regression/complexlogicor
2016-05-18 01:32:58,311 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:58,311 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:58,380 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 01:32:58,397 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.045
2016-05-18 01:32:58,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:58,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:32:58,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:58,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:32:58,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:58,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting
2016-05-18 01:32:58,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:58,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] parallel execution of steps is enabled: false
2016-05-18 01:32:58,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executing total steps: 1
2016-05-18 01:32:58,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] allocating management threads: 1
2016-05-18 01:32:58,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting step: (1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,477 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [split] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:58,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tez session mode enabled: true
2016-05-18 01:32:58,520 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:58,521 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:58,521 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:58,531 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001 doesn't exist and is created
2016-05-18 01:32:58,568 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:58,568 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:58,570 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd
2016-05-18 01:32:58,589 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535178521_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:58,589 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:58,590 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:58,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:58,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:58,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:58,596 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:58,597 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:58,603 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:58,603 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:58,603 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:58,604 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:58,604 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:58,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:58,604 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:58,605 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:58,605 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:58,605 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:58,606 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:58,606 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:58,615 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:58,617 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:58,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:58,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir
2016-05-18 01:32:58,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/history.txt.appattempt_1463535178521_0001_000000, maxErrors=10
2016-05-18 01:32:58,618 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:58,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535178521_0001_000000, appSubmitTime=1463535178588, launchTime=1463535178588
2016-05-18 01:32:58,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:58,641 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54229
2016-05-18 01:32:58,644 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535178521
2016-05-18 01:32:58,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535178521_0001_000000, startTime=1463535178652
2016-05-18 01:32:58,652 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:58,669 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:58,669 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:58,669 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001, dagName=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,700 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,701 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:58,702 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535178521_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/dag_1463535178521_0001_1.dot
2016-05-18 01:32:58,704 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001_wd/localmode-log-dir/dag_1463535178521_0001_1-tez-dag.pb.txt
2016-05-18 01:32:58,710 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535178521_0001_1
2016-05-18 01:32:58,710 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,711 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58 Running Dag: dag_1463535178521_0001_1
2016-05-18 01:32:58,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535178521_0001_1, submitTime=1463535178700
2016-05-18 01:32:58,731 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001/tez-conf.pb" } size: 40938 timestamp: 1463535178000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:58,732 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,732 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,732 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:58,733 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:58,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535178521_0001_1, initTime=1463535178733
2016-05-18 01:32:58,737 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:58,738 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:58,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_STARTED]: dagID=dag_1463535178521_0001_1, startTime=1463535178739
2016-05-18 01:32:58,739 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:58,739 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E0FD00F704B74F66B6AC601C3AA66EA4 : {E1A3E10480924DA68FAE85720A4CADB7={InputName=E1A3E10480924DA68FAE85720A4CADB7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:58,740 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E1A3E10480924DA68FAE85720A4CADB7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:58,740 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,740 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001, dagName=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate
2016-05-18 01:32:58,741 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,743 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:58,744 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E1A3E10480924DA68FAE85720A4CADB7 on vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,754 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E1A3E10480924DA68FAE85720A4CADB7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:58,759 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:58,759 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:32:58,760 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:58,760 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:32:58,762 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E1A3E10480924DA68FAE85720A4CADB7 on vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,764 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:58,765 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E1A3E10480924DA68FAE85720A4CADB7=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:58,766 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] parallelism set to 1
2016-05-18 01:32:58,767 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:58,768 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], numAdditionalOutputs=1
2016-05-18 01:32:58,768 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FFDAAF19BA3149E2BE99E4ACA1A46D82, vertex=vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:58,776 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E0FD00F704B74F66B6AC601C3AA66EA4:FFDAAF19BA3149E2BE99E4ACA1A46D82 using oldmapred API
2016-05-18 01:32:58,776 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FFDAAF19BA3149E2BE99E4ACA1A46D82, vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:58,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, vertexId=vertex_1463535178521_0001_1_00, initRequestedTime=1463535178739, initedTime=1463535178768, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:58,780 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] with distanceFromRoot: 0
2016-05-18 01:32:58,780 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:58,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535178521_0001_1_00, startRequestedTime=1463535178744, startedTime=1463535178781
2016-05-18 01:32:58,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535178521_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:58,782 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:58,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] submitted tez dag to app master: application_1463535178521_0001, with dag id: dag_1463535178521_0001_1
2016-05-18 01:32:58,783 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E0FD00F704B74F66B6AC601C3AA66EA4
2016-05-18 01:32:58,783 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_STARTED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskId=task_1463535178521_0001_1_00_000000, scheduledTime=1463535178784, launchTime=1463535178784
2016-05-18 01:32:58,785 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,789 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:58,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535178521_0001_00_000001, launchTime=1463535178794
2016-05-18 01:32:58,800 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,801 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535178521_0001_1_00_000000_0] started. Is using containerId: [container_1463535178521_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:58,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, startTime=1463535178801, containerId=container_1463535178521_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:58,803 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535178521_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58 Starting to run new task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,804 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,805 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:58,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, VertexName: E0FD00F704B74F66B6AC601C3AA66EA4, VertexParallelism: 1, TaskAttemptID:attempt_1463535178521_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1A3E10480924DA68FAE85720A4CADB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:32:58,806 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:58,808 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,812 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535178521_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:58,813 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535178521_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:58,819 INFO input.MRInput (MRInput.java:initialize(440)) - E1A3E10480924DA68FAE85720A4CADB7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:58,847 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,850 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:58,864 INFO output.MROutput (MROutput.java:initialize(421)) - FFDAAF19BA3149E2BE99E4ACA1A46D82: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:32:58,867 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, ordinal: 0
2016-05-18 01:32:58,867 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, mem on start (mb), free: 258, total: 281, max: 672
2016-05-18 01:32:58,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:58,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:58,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:58,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:58,867 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E1A3E10480924DA68FAE85720A4CADB7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FFDAAF19BA3149E2BE99E4ACA1A46D82:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:32:58,867 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:58,869 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:58,870 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:32:58,870 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E1A3E10480924DA68FAE85720A4CADB7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:32:58,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:58,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E1A3E10480924DA68FAE85720A4CADB7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:58,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:58,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:58,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:58,871 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,874 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:32:58,874 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E1A3E10480924DA68FAE85720A4CADB7 initialized RecordReader from event
2016-05-18 01:32:58,878 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,881 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: E1A3E10480924DA68FAE85720A4CADB7
2016-05-18 01:32:58,882 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"], id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:58,882 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:58,883 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:32:58,883 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:32:58,888 INFO output.MROutput (MROutput.java:close(526)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 closed
2016-05-18 01:32:58,889 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535178521_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:32:58,890 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,890 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0FD00F704B74F66B6AC601C3AA66EA4, mem on close (mb), free: 255, total: 281, max: 672
2016-05-18 01:32:58,890 INFO output.MROutput (MROutput.java:close(526)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 closed
2016-05-18 01:32:58,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:32:58 Completed running task attempt: attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,892 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:32:58,892 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535178521_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:32:58,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535178521_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:32:58,895 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535178521_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,895 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:32:58,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskAttemptId=attempt_1463535178521_0001_1_00_000000_0, creationTime=1463535178785, allocationTime=1463535178793, startTime=1463535178801, finishTime=1463535178895, timeTaken=94, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:32:58,895 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:32:58,896 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,896 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535178521_0001_1_00_000000_0
2016-05-18 01:32:58,896 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:TASK_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, taskId=task_1463535178521_0001_1_00_000000, startTime=1463535178801, finishTime=1463535178896, timeTaken=95, status=SUCCEEDED, successfulAttemptID=attempt_1463535178521_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:32:58,899 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:32:58,899 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4]
2016-05-18 01:32:58,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:VERTEX_FINISHED]: vertexName=E0FD00F704B74F66B6AC601C3AA66EA4, vertexId=vertex_1463535178521_0001_1_00, initRequestedTime=1463535178739, initedTime=1463535178768, startRequestedTime=1463535178744, startedTime=1463535178781, finishTime=1463535178899, timeTaken=118, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535178801, firstTasksToStart=[ task_1463535178521_0001_1_00_000000 ], lastTaskFinishTime=1463535178895, lastTasksToFinish=[ task_1463535178521_0001_1_00_000000 ], minTaskDuration=94, maxTaskDuration=94, avgTaskDuration=94.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535178521_0001_1_00_000000 ], longestDurationTasks=[ task_1463535178521_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:32:58,901 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:32:58,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535178521_0001_00_000001, stoppedTime=1463535178901, exitStatus=0
2016-05-18 01:32:58,901 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:32:58,901 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,902 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535178521_0001_1
2016-05-18 01:32:58,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535178521_0001_1
2016-05-18 01:32:58,902 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: FFDAAF19BA3149E2BE99E4ACA1A46D82 for vertex: vertex_1463535178521_0001_1_00 [E0FD00F704B74F66B6AC601C3AA66EA4], outputName: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:58,902 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:32:58,904 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535178521_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:32:58,907 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535178521_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:32:58,907 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535178521_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:32:58,908 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535178521_0001_00_000001, Exiting
2016-05-18 01:32:58,908 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,908 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535178521_0001_00_000001
2016-05-18 01:32:58,910 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535178521_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:32:58,910 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535178521_0001_00_000001 completed successfully
2016-05-18 01:32:58,918 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:FFDAAF19BA3149E2BE99E4ACA1A46D82 of vertex/vertexGroup:E0FD00F704B74F66B6AC601C3AA66EA4 isVertexGroupOutput:false
2016-05-18 01:32:58,918 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:32:58,920 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535178521_0001_1, queueSize=0
2016-05-18 01:32:58,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535178521_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535178521_0001_1, startTime=1463535178738, finishTime=1463535178918, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=470, AM_GC_TIME_MILLIS=27, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535178884, Process_Duration=5, Process_End_Time=1463535178889, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:32:58,921 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535178521_0001_1 finished with state: SUCCEEDED
2016-05-18 01:32:58,921 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535178521_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:32:58 Completed Dag: dag_1463535178521_0001_1
2016-05-18 01:32:58,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535178521_0001_1, dagState=SUCCEEDED
2016-05-18 01:32:58,922 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535178521_0001_1_post
2016-05-18 01:32:58,922 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:58,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:32:58,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:32:58,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, with id=dag_1463535178521_0001_1
2016-05-18 01:32:58,923 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F] split/(1/1) ...oupSplitPipe/intermediate, with id=dag_1463535178521_0001_1
2016-05-18 01:32:58,929 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...oupSplitPipe/intermediate, applicationId=application_1463535178521_0001
2016-05-18 01:32:58,933 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:32:58,933 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:32:58,933 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:32:58,933 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:32:58,933 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:32:58,934 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:32:58,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:32:58,936 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:32:58,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:32:58,937 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,937 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:32:58,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:32:58,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:32:58,939 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:32:58,943 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/55676E151DB14CAF83DBE0F4BD571D2C/A6AE994CAA9441B4B908D90C2508AD5F/.tez/application_1463535178521_0001
2016-05-18 01:32:58,945 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/_temporary
2016-05-18 01:32:58,946 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] completed in: 00:00.474
2016-05-18 01:32:58,989 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:32:58,990 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:32:59,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.083
2016-05-18 01:32:59,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.105
2016-05-18 01:32:59,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:32:59,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:32:59,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:32:59,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:59,209 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:32:59,233 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit
2016-05-18 01:32:59,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] starting
2016-05-18 01:32:59,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"]
2016-05-18 01:32:59,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] parallel execution of steps is enabled: false
2016-05-18 01:32:59,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] executing total steps: 1
2016-05-18 01:32:59,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] allocating management threads: 1
2016-05-18 01:32:59,385 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] starting step: (1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,385 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lhs*rhs] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:32:59,402 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] tez session mode enabled: true
2016-05-18 01:32:59,406 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:59,407 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:32:59,407 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:32:59,417 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001 doesn't exist and is created
2016-05-18 01:32:59,442 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:32:59,442 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:32:59,443 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd
2016-05-18 01:32:59,459 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535179407_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:32:59,460 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:59,460 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:32:59,462 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:32:59,462 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:32:59,463 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:32:59,463 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:32:59,464 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:32:59,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:32:59,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:32:59,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:32:59,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:32:59,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:32:59,468 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:32:59,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:32:59,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:32:59,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:59,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:32:59,469 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:32:59,469 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:32:59,473 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:32:59,474 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:32:59,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:32:59,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir
2016-05-18 01:32:59,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/history.txt.appattempt_1463535179407_0001_000000, maxErrors=10
2016-05-18 01:32:59,476 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:32:59,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535179407_0001_000000, appSubmitTime=1463535179459, launchTime=1463535179459
2016-05-18 01:32:59,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:32:59,492 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33492
2016-05-18 01:32:59,492 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535179407
2016-05-18 01:32:59,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535179407_0001_000000, startTime=1463535179496
2016-05-18 01:32:59,496 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:32:59,542 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:32:59,543 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:32:59,543 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001, dagName=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,569 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,569 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:32:59,571 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535179407_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/dag_1463535179407_0001_1.dot
2016-05-18 01:32:59,573 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-log-dir/dag_1463535179407_0001_1-tez-dag.pb.txt
2016-05-18 01:32:59,589 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535179407_0001_1
2016-05-18 01:32:59,589 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,589 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59 Running Dag: dag_1463535179407_0001_1
2016-05-18 01:32:59,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535179407_0001_1, submitTime=1463535179569
2016-05-18 01:32:59,607 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,607 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,608 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,609 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,609 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,609 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,610 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001/tez-conf.pb" } size: 40938 timestamp: 1463535179000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:32:59,611 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,611 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:32:59,620 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:32:59,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535179407_0001_1, initTime=1463535179620
2016-05-18 01:32:59,621 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:32:59,622 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:32:59,622 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001, dagName=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit
2016-05-18 01:32:59,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_STARTED]: dagID=dag_1463535179407_0001_1, startTime=1463535179622
2016-05-18 01:32:59,622 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BA80601CC7DA4457869BE0F2F8B5EC29 : {FFDAAF19BA3149E2BE99E4ACA1A46D82={InputName=FFDAAF19BA3149E2BE99E4ACA1A46D82}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FFDAAF19BA3149E2BE99E4ACA1A46D82, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,623 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,624 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:59,624 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B1A06BDE26DF401DB6A2AD73562926CF : {FFDAAF19BA3149E2BE99E4ACA1A46D82={InputName=FFDAAF19BA3149E2BE99E4ACA1A46D82}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:32:59,624 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FFDAAF19BA3149E2BE99E4ACA1A46D82, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:32:59,625 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,625 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,626 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:32:59,626 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,626 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,629 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,631 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FFDAAF19BA3149E2BE99E4ACA1A46D82 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:59,633 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FFDAAF19BA3149E2BE99E4ACA1A46D82 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:32:59,635 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:32:59,637 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,638 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:32:59,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:32:59,639 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], numAdditionalOutputs=1
2016-05-18 01:32:59,639 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=B5195B2CFFEB4747B400F2242B36B10F, vertex=vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:32:59,642 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AAD10383766F45E39B8107B0CA5AC336:B5195B2CFFEB4747B400F2242B36B10F using oldmapred API
2016-05-18 01:32:59,642 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=B5195B2CFFEB4747B400F2242B36B10F, vertexName=AAD10383766F45E39B8107B0CA5AC336, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:32:59,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, vertexId=vertex_1463535179407_0001_1_02, initRequestedTime=1463535179626, initedTime=1463535179639, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:32:59,644 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:32:59,678 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:59,678 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:32:59,678 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:32:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:32:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:32:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:32:59,680 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,682 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 on vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,684 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,688 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,689 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FFDAAF19BA3149E2BE99E4ACA1A46D82=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:59,690 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FFDAAF19BA3149E2BE99E4ACA1A46D82=forAllWorkUnits=true, update=[1]}
2016-05-18 01:32:59,692 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] parallelism set to 1
2016-05-18 01:32:59,692 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] submitted tez dag to app master: application_1463535179407_0001, with dag id: dag_1463535179407_0001_1
2016-05-18 01:32:59,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, vertexId=vertex_1463535179407_0001_1_01, initRequestedTime=1463535179624, initedTime=1463535179693, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:59,693 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] with distanceFromRoot: 0
2016-05-18 01:32:59,693 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:59,692 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] parallelism set to 1
2016-05-18 01:32:59,694 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:32:59,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_01, startRequestedTime=1463535179626, startedTime=1463535179693
2016-05-18 01:32:59,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:59,695 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, vertexId=vertex_1463535179407_0001_1_00, initRequestedTime=1463535179623, initedTime=1463535179695, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:32:59,696 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] with distanceFromRoot: 0
2016-05-18 01:32:59,696 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:32:59,696 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:32:59,696 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,696 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535179407_0001_1_01 for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 1 numSources: 2
2016-05-18 01:32:59,697 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 1 numSources: 2
2016-05-18 01:32:59,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_00, startRequestedTime=1463535179626, startedTime=1463535179697
2016-05-18 01:32:59,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535179407_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:32:59,697 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,698 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535179407_0001_1_00 for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] numStartedSources: 2 numSources: 2
2016-05-18 01:32:59,698 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] with distanceFromRoot: 1
2016-05-18 01:32:59,699 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:32:59,699 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskId=task_1463535179407_0001_1_01_000000, scheduledTime=1463535179698, launchTime=1463535179698
2016-05-18 01:32:59,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535179407_0001_1_02, startRequestedTime=1463535179698, startedTime=1463535179701
2016-05-18 01:32:59,701 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:32:59,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskId=task_1463535179407_0001_1_00_000000, scheduledTime=1463535179704, launchTime=1463535179704
2016-05-18 01:32:59,704 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:32:59,704 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:32:59,706 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:32:59,720 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: AAD10383766F45E39B8107B0CA5AC336 with 0 source tasks and 1 pending tasks
2016-05-18 01:32:59,723 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B1A06BDE26DF401DB6A2AD73562926CF in vertex: AAD10383766F45E39B8107B0CA5AC336 numBipartiteSourceTasks: 1
2016-05-18 01:32:59,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000001, launchTime=1463535179721
2016-05-18 01:32:59,724 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: BA80601CC7DA4457869BE0F2F8B5EC29 in vertex: AAD10383766F45E39B8107B0CA5AC336 numBipartiteSourceTasks: 2
2016-05-18 01:32:59,726 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000001
2016-05-18 01:32:59,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000002, launchTime=1463535179723
2016-05-18 01:32:59,728 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,728 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_01_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:32:59,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, startTime=1463535179727, containerId=container_1463535179407_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:32:59,730 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000002
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,731 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,731 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,731 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:59,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: B1A06BDE26DF401DB6A2AD73562926CF, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AAD10383766F45E39B8107B0CA5AC336, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:32:59,733 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:59,734 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_00_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:32:59,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, startTime=1463535179733, containerId=container_1463535179407_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:32:59,736 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,736 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,738 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535179407_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:32:59,738 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:59 Starting to run new task attempt: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,739 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,739 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:32:59,740 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:32:59,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: BA80601CC7DA4457869BE0F2F8B5EC29, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FFDAAF19BA3149E2BE99E4ACA1A46D82, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AAD10383766F45E39B8107B0CA5AC336, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:32:59,740 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:32:59,741 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,744 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535179407_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:32:59,745 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:32:59,755 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:59,777 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:32:59,783 INFO input.MRInput (MRInput.java:initialize(440)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:59,783 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,786 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, ordinal: 1
2016-05-18 01:32:59,788 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, mem on start (mb), free: 348, total: 388, max: 672
2016-05-18 01:32:59,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:59,787 INFO input.MRInput (MRInput.java:initialize(440)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:32:59,789 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:59,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:59,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:59,791 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FFDAAF19BA3149E2BE99E4ACA1A46D82:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AAD10383766F45E39B8107B0CA5AC336:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:32:59,792 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:59,792 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:59,794 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:32:59,794 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AAD10383766F45E39B8107B0CA5AC336:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:32:59,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:59,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:59,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:59,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:59,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:59,801 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:32:59,806 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:32:59,808 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 initialized RecordReader from event
2016-05-18 01:32:59,807 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,811 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, ordinal: 0
2016-05-18 01:32:59,811 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, mem on start (mb), free: 346, total: 388, max: 672
2016-05-18 01:32:59,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:32:59,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:32:59,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:32:59,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:32:59,813 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FFDAAF19BA3149E2BE99E4ACA1A46D82:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AAD10383766F45E39B8107B0CA5AC336:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:32:59,813 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:32:59,814 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:32:59,814 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:32:59,815 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FFDAAF19BA3149E2BE99E4ACA1A46D82:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AAD10383766F45E39B8107B0CA5AC336:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:32:59,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:32:59,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FFDAAF19BA3149E2BE99E4ACA1A46D82 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:32:59,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:32:59,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:32:59,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:32:59,818 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:32:59,820 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:32:59,821 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FFDAAF19BA3149E2BE99E4ACA1A46D82 initialized RecordReader from event
2016-05-18 01:32:59,824 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:32:59,827 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"] streamed: true, id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,827 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"] streamed: true, id: FFDAAF19BA3149E2BE99E4ACA1A46D82
2016-05-18 01:32:59,827 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']], id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,828 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,827 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']], id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,829 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:32:59,875 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AAD10383766F45E39B8107B0CA5AC336 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4a1dde79, comparator=cascading.tuple.hadoop.util.TupleComparator@6ef8fff2, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:32:59,875 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AAD10383766F45E39B8107B0CA5AC336 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4a1dde79, comparator=cascading.tuple.hadoop.util.TupleComparator@1d6ba1c9, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:32:59,881 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false
2016-05-18 01:32:59,881 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false
2016-05-18 01:33:00,224 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:00,225 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:00,225 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:00,228 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AAD10383766F45E39B8107B0CA5AC336: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:00,235 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:33:00,237 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:00,238 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:00,238 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:00,238 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AAD10383766F45E39B8107B0CA5AC336: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:00,238 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AAD10383766F45E39B8107B0CA5AC336: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:00,240 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate"]
2016-05-18 01:33:00,240 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:00,243 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00,244 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA80601CC7DA4457869BE0F2F8B5EC29, mem on close (mb), free: 256, total: 475, max: 672
2016-05-18 01:33:00,245 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AAD10383766F45E39B8107B0CA5AC336: Starting flush of map output
2016-05-18 01:33:00,245 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AAD10383766F45E39B8107B0CA5AC336: Span0.length = 5, perItem = 15
2016-05-18 01:33:00,246 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,246 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A06BDE26DF401DB6A2AD73562926CF, mem on close (mb), free: 256, total: 475, max: 672
2016-05-18 01:33:00,247 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AAD10383766F45E39B8107B0CA5AC336: done sorting span=0, length=5, time=1
2016-05-18 01:33:00,248 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AAD10383766F45E39B8107B0CA5AC336: Starting flush of map output
2016-05-18 01:33:00,249 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AAD10383766F45E39B8107B0CA5AC336: Span0.length = 5, perItem = 9
2016-05-18 01:33:00,249 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AAD10383766F45E39B8107B0CA5AC336: done sorting span=0, length=5, time=0
2016-05-18 01:33:00,250 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AAD10383766F45E39B8107B0CA5AC336: Heap = SpanIterator<0:4> (span=Span[16777216,45]),
2016-05-18 01:33:00,250 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AAD10383766F45E39B8107B0CA5AC336: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:00,256 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AAD10383766F45E39B8107B0CA5AC336: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_00_000000_0_10012_0/file.out
2016-05-18 01:33:00,257 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AAD10383766F45E39B8107B0CA5AC336: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_01_000000_0_10010_0/file.out
2016-05-18 01:33:00,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,287 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:00,288 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:00,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:00,288 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:00,291 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:00,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskAttemptId=attempt_1463535179407_0001_1_01_000000_0, creationTime=1463535179699, allocationTime=1463535179718, startTime=1463535179727, finishTime=1463535180291, timeTaken=564, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,291 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,293 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,293 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:00,293 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:00,293 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,297 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,297 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,298 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:00,298 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,298 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000002
2016-05-18 01:33:00,299 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_01_000000_0
2016-05-18 01:33:00,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, taskId=task_1463535179407_0001_1_01_000000, startTime=1463535179727, finishTime=1463535180299, timeTaken=572, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:00,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskAttemptId=attempt_1463535179407_0001_1_00_000000_0, creationTime=1463535179704, allocationTime=1463535179719, startTime=1463535179733, finishTime=1463535180301, timeTaken=568, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,304 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,305 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:33:00,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=B1A06BDE26DF401DB6A2AD73562926CF, vertexId=vertex_1463535179407_0001_1_01, initRequestedTime=1463535179624, initedTime=1463535179693, startRequestedTime=1463535179626, startedTime=1463535179693, finishTime=1463535180305, timeTaken=612, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=45, OUTPUT_BYTES_WITH_OVERHEAD=61, OUTPUT_BYTES_PHYSICAL=65, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180240, Process_Duration=6, Process_End_Time=1463535180246, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535179727, firstTasksToStart=[ task_1463535179407_0001_1_01_000000 ], lastTaskFinishTime=1463535180291, lastTasksToFinish=[ task_1463535179407_0001_1_01_000000 ], minTaskDuration=564, maxTaskDuration=564, avgTaskDuration=564.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_01_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,307 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000001, stoppedTime=1463535180307, exitStatus=0
2016-05-18 01:33:00,308 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_00_000000_0
2016-05-18 01:33:00,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, taskId=task_1463535179407_0001_1_00_000000, startTime=1463535179733, finishTime=1463535180308, timeTaken=575, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:00,312 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] attempt: attempt_1463535179407_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:00,312 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,312 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,313 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000002, stoppedTime=1463535180313, exitStatus=0
2016-05-18 01:33:00,313 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,313 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:33:00,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=BA80601CC7DA4457869BE0F2F8B5EC29, vertexId=vertex_1463535179407_0001_1_00, initRequestedTime=1463535179623, initedTime=1463535179695, startRequestedTime=1463535179626, startedTime=1463535179697, finishTime=1463535180313, timeTaken=616, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535180238, Process_Duration=5, Process_End_Time=1463535180243, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535179733, firstTasksToStart=[ task_1463535179407_0001_1_00_000000 ], lastTaskFinishTime=1463535180301, lastTasksToFinish=[ task_1463535179407_0001_1_00_000000 ], minTaskDuration=568, maxTaskDuration=568, avgTaskDuration=568.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_00_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,315 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,316 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] attempt: attempt_1463535179407_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:00,316 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,316 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,316 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,316 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AAD10383766F45E39B8107B0CA5AC336
2016-05-18 01:33:00,317 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B1A06BDE26DF401DB6A2AD73562926CF, destinationVertexName=AAD10383766F45E39B8107B0CA5AC336. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:00,317 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=BA80601CC7DA4457869BE0F2F8B5EC29, destinationVertexName=AAD10383766F45E39B8107B0CA5AC336. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:00,317 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_STARTED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskId=task_1463535179407_0001_1_02_000000, scheduledTime=1463535180318, launchTime=1463535180318
2016-05-18 01:33:00,319 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535179407_0001_00_000003, launchTime=1463535180321
2016-05-18 01:33:00,323 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,324 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,324 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,324 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535179407_0001_1_02_000000_0] started. Is using containerId: [container_1463535179407_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:00,324 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000001, Exiting
2016-05-18 01:33:00,324 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,327 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000001
2016-05-18 01:33:00,326 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, startTime=1463535180324, containerId=container_1463535179407_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:00,332 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,332 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000002: 33 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,332 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000002, Exiting
2016-05-18 01:33:00,332 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000002
2016-05-18 01:33:00 Starting to run new task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,332 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000002
2016-05-18 01:33:00,332 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,332 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,333 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:00,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000001 completed successfully
2016-05-18 01:33:00,334 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,334 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000002 completed successfully
2016-05-18 01:33:00,334 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, VertexName: AAD10383766F45E39B8107B0CA5AC336, VertexParallelism: 1, TaskAttemptID:attempt_1463535179407_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1A06BDE26DF401DB6A2AD73562926CF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=BA80601CC7DA4457869BE0F2F8B5EC29, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B5195B2CFFEB4747B400F2242B36B10F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:00,335 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:00,336 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,347 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535179407_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,347 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:00,347 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535179407_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:00,355 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:00,356 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:00,360 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, ordinal: 2
2016-05-18 01:33:00,360 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, mem on start (mb), free: 244, total: 475, max: 672
2016-05-18 01:33:00,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:00,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:00,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:00,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:00,366 INFO output.MROutput (MROutput.java:initialize(421)) - B5195B2CFFEB4747B400F2242B36B10F: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:00,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:00,367 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B5195B2CFFEB4747B400F2242B36B10F:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B1A06BDE26DF401DB6A2AD73562926CF:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [BA80601CC7DA4457869BE0F2F8B5EC29:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:00,367 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:00,367 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:00,367 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:00,368 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B5195B2CFFEB4747B400F2242B36B10F:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B1A06BDE26DF401DB6A2AD73562926CF:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [BA80601CC7DA4457869BE0F2F8B5EC29:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:00,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:00,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B1A06BDE26DF401DB6A2AD73562926CF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:00,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BA80601CC7DA4457869BE0F2F8B5EC29 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:00,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:33:00,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:33:00,369 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B1A06BDE26DF401DB6A2AD73562926CF: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:00,374 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B1A06BDE26DF401DB6A2AD73562926CF: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:00,376 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - BA80601CC7DA4457869BE0F2F8B5EC29: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:00,376 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - BA80601CC7DA4457869BE0F2F8B5EC29: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:00,392 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: BA80601CC7DA4457869BE0F2F8B5EC29 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:00,392 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B1A06BDE26DF401DB6A2AD73562926CF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:00,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:00,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:00,398 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,404 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']]
2016-05-18 01:33:00,405 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A06BDE26DF401DB6A2AD73562926CF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:00,405 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BA80601CC7DA4457869BE0F2F8B5EC29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:00,408 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] streamed: true, id: 2A5D39D18B77441495EE822D19BE7932
2016-05-18 01:33:00,408 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"], id: B5195B2CFFEB4747B400F2242B36B10F
2016-05-18 01:33:00,409 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit"]
2016-05-18 01:33:00,409 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932, for 2 inputs
2016-05-18 01:33:00,411 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lhs*rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num2']] 2A5D39D18B77441495EE822D19BE7932, for 2 inputs
2016-05-18 01:33:00,414 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535179407_0001_1_00_000000_0_10012} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535180414, TimeTaken=2, Rate=0.05 MB/s
2016-05-18 01:33:00,414 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,414 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535179407_0001_1_01_000000_0_10010} to DISK_DIRECT, csize=65, dsize=61, EndTime=1463535180414, TimeTaken=4, Rate=0.02 MB/s
2016-05-18 01:33:00,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: BA80601CC7DA4457869BE0F2F8B5EC29, wasInterrupted=false
2016-05-18 01:33:00,415 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,416 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,416 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,417 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B1A06BDE26DF401DB6A2AD73562926CF, wasInterrupted=false
2016-05-18 01:33:00,416 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:00,417 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:00,418 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:00,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_01_000000_0_10010/file.out, len=65, isLocal=true
2016-05-18 01:33:00,419 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:00,419 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:00,420 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 65 bytes from disk
2016-05-18 01:33:00,420 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:00,422 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001_wd/localmode-local-dir/output/attempt_1463535179407_0001_1_00_000000_0_10012/file.out, len=95, isLocal=true
2016-05-18 01:33:00,422 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:00,422 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:00,424 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:00,424 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:00,430 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 48 bytes
2016-05-18 01:33:00,430 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:00,430 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,430 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B1A06BDE26DF401DB6A2AD73562926CF: Shuffle Runner thread complete
2016-05-18 01:33:00,430 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, all 2 inputs ready in: 00:00:00.004
2016-05-18 01:33:00,431 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,431 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - BA80601CC7DA4457869BE0F2F8B5EC29: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:00,432 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - BA80601CC7DA4457869BE0F2F8B5EC29: Shuffle Runner thread complete
2016-05-18 01:33:00,435 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B1A06BDE26DF401DB6A2AD73562926CF: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:00,450 INFO output.MROutput (MROutput.java:close(526)) - B5195B2CFFEB4747B400F2242B36B10F closed
2016-05-18 01:33:00,450 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535179407_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:00,451 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,451 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD10383766F45E39B8107B0CA5AC336, mem on close (mb), free: 235, total: 475, max: 672
2016-05-18 01:33:00,452 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B1A06BDE26DF401DB6A2AD73562926CF
2016-05-18 01:33:00,452 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A06BDE26DF401DB6A2AD73562926CF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:00,452 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: BA80601CC7DA4457869BE0F2F8B5EC29
2016-05-18 01:33:00,452 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BA80601CC7DA4457869BE0F2F8B5EC29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:00,453 INFO output.MROutput (MROutput.java:close(526)) - B5195B2CFFEB4747B400F2242B36B10F closed
2016-05-18 01:33:00,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:00 Completed running task attempt: attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,454 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:00,454 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535179407_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:00,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535179407_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:00,456 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535179407_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,456 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:00,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskAttemptId=attempt_1463535179407_0001_1_02_000000_0, creationTime=1463535180319, allocationTime=1463535180320, startTime=1463535180324, finishTime=1463535180456, timeTaken=132, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:00,458 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535179407_0001_1_02_000000_0
2016-05-18 01:33:00,456 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:00,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,458 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:TASK_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, taskId=task_1463535179407_0001_1_02_000000, startTime=1463535180324, finishTime=1463535180458, timeTaken=134, status=SUCCEEDED, successfulAttemptID=attempt_1463535179407_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:00,459 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:00,460 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336]
2016-05-18 01:33:00,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:VERTEX_FINISHED]: vertexName=AAD10383766F45E39B8107B0CA5AC336, vertexId=vertex_1463535179407_0001_1_02, initRequestedTime=1463535179626, initedTime=1463535179639, startRequestedTime=1463535179698, startedTime=1463535179701, finishTime=1463535180460, timeTaken=759, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535180430, Process_Duration=21, Process_End_Time=1463535180451, Read_Duration=3, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535180324, firstTasksToStart=[ task_1463535179407_0001_1_02_000000 ], lastTaskFinishTime=1463535180456, lastTasksToFinish=[ task_1463535179407_0001_1_02_000000 ], minTaskDuration=132, maxTaskDuration=132, avgTaskDuration=132.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535179407_0001_1_02_000000 ], longestDurationTasks=[ task_1463535179407_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:00,461 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:00,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535179407_0001_00_000003, stoppedTime=1463535180462, exitStatus=0
2016-05-18 01:33:00,462 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:00,462 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,462 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535179407_0001_1_00 [BA80601CC7DA4457869BE0F2F8B5EC29]
2016-05-18 01:33:00,462 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535179407_0001_1_01 [B1A06BDE26DF401DB6A2AD73562926CF]
2016-05-18 01:33:00,462 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535179407_0001_1
2016-05-18 01:33:00,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535179407_0001_1
2016-05-18 01:33:00,463 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: B5195B2CFFEB4747B400F2242B36B10F for vertex: vertex_1463535179407_0001_1_02 [AAD10383766F45E39B8107B0CA5AC336], outputName: B5195B2CFFEB4747B400F2242B36B10F
2016-05-18 01:33:00,465 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:00,467 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535179407_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:00,469 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535179407_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:00,469 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535179407_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:00,470 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535179407_0001_00_000003, Exiting
2016-05-18 01:33:00,470 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,470 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535179407_0001_00_000003
2016-05-18 01:33:00,471 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535179407_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:00,471 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535179407_0001_00_000003 completed successfully
2016-05-18 01:33:00,476 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:B5195B2CFFEB4747B400F2242B36B10F of vertex/vertexGroup:AAD10383766F45E39B8107B0CA5AC336 isVertexGroupOutput:false
2016-05-18 01:33:00,476 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:00,479 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535179407_0001_1, queueSize=0
2016-05-18 01:33:00,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535179407_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535179407_0001_1, startTime=1463535179622, finishTime=1463535180476, timeTaken=854, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=2220, AM_GC_TIME_MILLIS=353, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=120, OUTPUT_BYTES_WITH_OVERHEAD=152, OUTPUT_BYTES_PHYSICAL=160, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=160, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=160, SHUFFLE_BYTES_DECOMPRESSED=152, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=160, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=87, MERGE_PHASE_TIME=115, FIRST_EVENT_RECEIVED=65, LAST_EVENT_RECEIVED=65, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605540908, Process_Duration=32, Process_End_Time=4390605540940, Read_Duration=10, Tuples_Read=20, Tuples_Written=15, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-18 01:33:00,482 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535179407_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:00,483 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535179407_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:00 Completed Dag: dag_1463535179407_0001_1
2016-05-18 01:33:00,483 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535179407_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:00,483 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535179407_0001_1_post
2016-05-18 01:33:00,483 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:00,483 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:00,484 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:00,484 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, with id=dag_1463535179407_0001_1
2016-05-18 01:33:00,484 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC] lhs*rhs/(1/1) ...oupSplitPipe/cogroupsplit, with id=dag_1463535179407_0001_1
2016-05-18 01:33:00,491 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...oupSplitPipe/cogroupsplit, applicationId=application_1463535179407_0001
2016-05-18 01:33:00,493 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:00,493 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:00,494 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:00,494 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:00,494 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:00,494 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:00,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:00,495 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:00,495 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:00,495 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:00,495 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:00,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:00,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:00,499 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:00,504 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A31C7581C7C541ABA6787763F76C3902/F6F9EB4640ED4B01AF4BD33CECE70AAC/.tez/application_1463535179407_0001
2016-05-18 01:33:00,505 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit/_temporary
2016-05-18 01:33:00,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs*rhs] completed in: 00:01.122
2016-05-18 01:33:00,619 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/cogroupsplit
2016-05-18 01:33:00,619 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCoGroupSplitPipe/intermediate
2016-05-18 01:33:00,662 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:00,663 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:00,738 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-18 01:33:00,746 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-18 01:33:00,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:00,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:00,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:00,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:00,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown
2016-05-18 01:33:00,872 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:00,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:00,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"]
2016-05-18 01:33:00,874 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:00,874 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:00,875 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:00,876 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:00,876 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:00,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:00,903 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:00,903 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:00,904 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:00,913 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001 doesn't exist and is created
2016-05-18 01:33:00,934 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:00,934 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:00,935 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd
2016-05-18 01:33:00,951 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535180903_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:00,952 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:00,952 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:00,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:00,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:00,956 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:00,959 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:00,959 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:00,960 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:00,961 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:00,961 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:00,961 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:00,961 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:00,965 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:00,966 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:00,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:00,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir
2016-05-18 01:33:00,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/history.txt.appattempt_1463535180903_0001_000000, maxErrors=10
2016-05-18 01:33:00,968 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:00,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535180903_0001_000000, appSubmitTime=1463535180951, launchTime=1463535180951
2016-05-18 01:33:00,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:00,981 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47230
2016-05-18 01:33:00,981 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535180903
2016-05-18 01:33:00,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535180903_0001_000000, startTime=1463535180985
2016-05-18 01:33:00,985 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:01,034 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:01,034 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:01,035 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001, dagName=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,065 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,066 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:01,067 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535180903_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/dag_1463535180903_0001_1.dot
2016-05-18 01:33:01,068 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-log-dir/dag_1463535180903_0001_1-tez-dag.pb.txt
2016-05-18 01:33:01,466 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535180903_0001_1
2016-05-18 01:33:01,466 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,466 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01 Running Dag: dag_1463535180903_0001_1
2016-05-18 01:33:01,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535180903_0001_1, submitTime=1463535181066
2016-05-18 01:33:01,484 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/tez-conf.pb" } size: 40938 timestamp: 1463535180000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:01,484 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,484 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:01,485 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001/tez-conf.pb" } size: 40938 timestamp: 1463535180000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:01,485 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,485 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:01,485 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:01,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535180903_0001_1, initTime=1463535181485
2016-05-18 01:33:01,486 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:01,489 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:01,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_STARTED]: dagID=dag_1463535180903_0001_1, startTime=1463535181489
2016-05-18 01:33:01,489 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:01,489 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001, dagName=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown
2016-05-18 01:33:01,489 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E654A931A7A841E8A650D30AAFDCC4A1 : {BACEA3E3751449D2B959BDEE247DF8FA={InputName=BACEA3E3751449D2B959BDEE247DF8FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:01,490 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: BACEA3E3751449D2B959BDEE247DF8FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:01,490 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,490 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,491 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:01,491 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,492 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: BACEA3E3751449D2B959BDEE247DF8FA on vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,494 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input BACEA3E3751449D2B959BDEE247DF8FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:01,496 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:01,497 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,497 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535180903_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:01,498 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], numAdditionalOutputs=1
2016-05-18 01:33:01,499 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A59D28D6A5304243ABA39F425135D08A, vertex=vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:01,501 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:01,501 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:01,501 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:01,502 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:01,502 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CAD7E907061C455184DD7CB2ED1DC6A1:A59D28D6A5304243ABA39F425135D08A using oldmapred API
2016-05-18 01:33:01,503 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A59D28D6A5304243ABA39F425135D08A, vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:01,503 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: BACEA3E3751449D2B959BDEE247DF8FA on vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, vertexId=vertex_1463535180903_0001_1_01, initRequestedTime=1463535181491, initedTime=1463535181498, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:01,504 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:01,505 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:01,506 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {BACEA3E3751449D2B959BDEE247DF8FA=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:01,506 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] parallelism set to 1
2016-05-18 01:33:01,506 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:01,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, vertexId=vertex_1463535180903_0001_1_00, initRequestedTime=1463535181489, initedTime=1463535181507, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:01,507 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] with distanceFromRoot: 0
2016-05-18 01:33:01,507 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:01,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535180903_0001_1_00, startRequestedTime=1463535181491, startedTime=1463535181507
2016-05-18 01:33:01,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535180903_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:01,508 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:01,508 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535180903_0001_1_00 for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] numStartedSources: 1 numSources: 1
2016-05-18 01:33:01,508 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] with distanceFromRoot: 1
2016-05-18 01:33:01,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535180903_0001_1_01, startRequestedTime=1463535181508, startedTime=1463535181508
2016-05-18 01:33:01,511 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:01,515 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,515 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_STARTED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskId=task_1463535180903_0001_1_00_000000, scheduledTime=1463535181515, launchTime=1463535181515
2016-05-18 01:33:01,516 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: CAD7E907061C455184DD7CB2ED1DC6A1 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:01,516 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,519 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E654A931A7A841E8A650D30AAFDCC4A1 in vertex: CAD7E907061C455184DD7CB2ED1DC6A1 numBipartiteSourceTasks: 1
2016-05-18 01:33:01,519 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:01,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535180903_0001_00_000001, launchTime=1463535181521
2016-05-18 01:33:01,522 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,524 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,524 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535180903_0001_1_00_000000_0] started. Is using containerId: [container_1463535180903_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:01,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, startTime=1463535181524, containerId=container_1463535180903_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535180903_0001, with dag id: dag_1463535180903_0001_1
2016-05-18 01:33:01,525 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,526 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,526 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:01,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, VertexName: E654A931A7A841E8A650D30AAFDCC4A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535180903_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BACEA3E3751449D2B959BDEE247DF8FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAD7E907061C455184DD7CB2ED1DC6A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:01,527 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:01,527 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,534 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:01,534 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535180903_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:01,535 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535180903_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:01,536 INFO input.MRInput (MRInput.java:initialize(440)) - BACEA3E3751449D2B959BDEE247DF8FA using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:01,536 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,543 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, ordinal: 0
2016-05-18 01:33:01,543 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, mem on start (mb), free: 360, total: 396, max: 672
2016-05-18 01:33:01,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:01,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:01,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:01,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:01,545 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[BACEA3E3751449D2B959BDEE247DF8FA:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CAD7E907061C455184DD7CB2ED1DC6A1:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:01,545 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:01,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:01,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:01,546 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[BACEA3E3751449D2B959BDEE247DF8FA:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CAD7E907061C455184DD7CB2ED1DC6A1:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:01,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:01,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BACEA3E3751449D2B959BDEE247DF8FA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:01,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:01,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:01,549 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,553 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:01,554 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:01,554 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - BACEA3E3751449D2B959BDEE247DF8FA initialized RecordReader from event
2016-05-18 01:33:01,555 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: BACEA3E3751449D2B959BDEE247DF8FA
2016-05-18 01:33:01,556 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{?}:NONE]], id: 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,556 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{?}:NONE]] 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,560 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - CAD7E907061C455184DD7CB2ED1DC6A1 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5c19b253, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@1b0a9662, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:01,560 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for CAD7E907061C455184DD7CB2ED1DC6A1: , UsingHashComparator=false
2016-05-18 01:33:01,572 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:01,573 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:01,573 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for CAD7E907061C455184DD7CB2ED1DC6A1: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:01,573 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - CAD7E907061C455184DD7CB2ED1DC6A1: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:01,574 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:01,574 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:01,577 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,577 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E654A931A7A841E8A650D30AAFDCC4A1, mem on close (mb), free: 258, total: 396, max: 672
2016-05-18 01:33:01,578 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - CAD7E907061C455184DD7CB2ED1DC6A1: Starting flush of map output
2016-05-18 01:33:01,578 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - CAD7E907061C455184DD7CB2ED1DC6A1: Span0.length = 5, perItem = 27
2016-05-18 01:33:01,578 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - CAD7E907061C455184DD7CB2ED1DC6A1: done sorting span=0, length=5, time=0
2016-05-18 01:33:01,579 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - CAD7E907061C455184DD7CB2ED1DC6A1: Heap = SpanIterator<0:4> (span=Span[16777216,135]),
2016-05-18 01:33:01,580 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - CAD7E907061C455184DD7CB2ED1DC6A1: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-local-dir/output/attempt_1463535180903_0001_1_00_000000_0_10019_0/file.out
2016-05-18 01:33:01,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,585 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:01,585 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535180903_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:01,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535180903_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:01,587 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535180903_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,587 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,588 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:01,588 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskAttemptId=attempt_1463535180903_0001_1_00_000000_0, creationTime=1463535181516, allocationTime=1463535181520, startTime=1463535181524, finishTime=1463535181587, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:01,588 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,589 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535180903_0001_1_00_000000_0
2016-05-18 01:33:01,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, taskId=task_1463535180903_0001_1_00_000000, startTime=1463535181524, finishTime=1463535181590, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463535180903_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:01,591 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:01,591 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_FINISHED]: vertexName=E654A931A7A841E8A650D30AAFDCC4A1, vertexId=vertex_1463535180903_0001_1_00, initRequestedTime=1463535181489, initedTime=1463535181507, startRequestedTime=1463535181491, startedTime=1463535181507, finishTime=1463535181591, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535181575, Process_Duration=2, Process_End_Time=1463535181577, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535181524, firstTasksToStart=[ task_1463535180903_0001_1_00_000000 ], lastTaskFinishTime=1463535181587, lastTasksToFinish=[ task_1463535180903_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535180903_0001_1_00_000000 ], longestDurationTasks=[ task_1463535180903_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:01,592 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:01,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535180903_0001_00_000001, stoppedTime=1463535181592, exitStatus=0
2016-05-18 01:33:01,593 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] attempt: attempt_1463535180903_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:01,593 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:01,593 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAD7E907061C455184DD7CB2ED1DC6A1
2016-05-18 01:33:01,593 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,594 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E654A931A7A841E8A650D30AAFDCC4A1, destinationVertexName=CAD7E907061C455184DD7CB2ED1DC6A1. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:01,594 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,594 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535180903_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:01,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_STARTED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskId=task_1463535180903_0001_1_01_000000, scheduledTime=1463535181595, launchTime=1463535181595
2016-05-18 01:33:01,595 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535180903_0001_00_000002, launchTime=1463535181597
2016-05-18 01:33:01,598 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,599 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535180903_0001_1_01_000000_0] started. Is using containerId: [container_1463535180903_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:01,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, startTime=1463535181599, containerId=container_1463535180903_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:01,600 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,600 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535180903_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:01,601 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:01,601 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535180903_0001_00_000001, Exiting
2016-05-18 01:33:01,601 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,601 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535180903_0001_00_000001
2016-05-18 01:33:01,602 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535180903_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:01 Starting to run new task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,602 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,602 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,602 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:01,602 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535180903_0001_00_000001 completed successfully
2016-05-18 01:33:01,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, VertexName: CAD7E907061C455184DD7CB2ED1DC6A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535180903_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E654A931A7A841E8A650D30AAFDCC4A1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A59D28D6A5304243ABA39F425135D08A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:01,603 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:01,604 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,609 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535180903_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,610 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535180903_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:01,611 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:01,614 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:01,625 INFO output.MROutput (MROutput.java:initialize(421)) - A59D28D6A5304243ABA39F425135D08A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:01,626 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, ordinal: 1
2016-05-18 01:33:01,626 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, mem on start (mb), free: 253, total: 396, max: 672
2016-05-18 01:33:01,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:01,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:01,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:01,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:01,627 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A59D28D6A5304243ABA39F425135D08A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E654A931A7A841E8A650D30AAFDCC4A1:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:01,627 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:01,627 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:01,628 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:01,629 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A59D28D6A5304243ABA39F425135D08A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E654A931A7A841E8A650D30AAFDCC4A1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:01,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:01,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E654A931A7A841E8A650D30AAFDCC4A1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:01,630 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E654A931A7A841E8A650D30AAFDCC4A1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:01,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:01,630 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E654A931A7A841E8A650D30AAFDCC4A1: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:01,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:01,632 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E654A931A7A841E8A650D30AAFDCC4A1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:01,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:01,634 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,636 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E654A931A7A841E8A650D30AAFDCC4A1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:01,638 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{?}:NONE]]
2016-05-18 01:33:01,639 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535180903_0001_1_00_000000_0_10019} to DISK_DIRECT, csize=155, dsize=151, EndTime=1463535181639, TimeTaken=1, Rate=0.15 MB/s
2016-05-18 01:33:01,639 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{?}:NONE]] streamed: true, id: 0D0222E4052A464EBAE45B11F76EE997
2016-05-18 01:33:01,639 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"], id: A59D28D6A5304243ABA39F425135D08A
2016-05-18 01:33:01,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,640 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown"]
2016-05-18 01:33:01,640 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{?}:NONE]] 0D0222E4052A464EBAE45B11F76EE997, for 1 inputs
2016-05-18 01:33:01,640 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:01,641 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E654A931A7A841E8A650D30AAFDCC4A1, wasInterrupted=false
2016-05-18 01:33:01,641 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:01,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:01,644 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001_wd/localmode-local-dir/output/attempt_1463535180903_0001_1_00_000000_0_10019/file.out, len=155, isLocal=true
2016-05-18 01:33:01,644 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 155 bytes from disk
2016-05-18 01:33:01,644 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:01,645 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:01,645 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 137 bytes
2016-05-18 01:33:01,645 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,645 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E654A931A7A841E8A650D30AAFDCC4A1: Shuffle Runner thread complete
2016-05-18 01:33:01,645 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, all 1 inputs ready in: 00:00:00.005
2016-05-18 01:33:01,646 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E654A931A7A841E8A650D30AAFDCC4A1: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:01,654 INFO output.MROutput (MROutput.java:close(526)) - A59D28D6A5304243ABA39F425135D08A closed
2016-05-18 01:33:01,655 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535180903_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:01,656 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,656 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAD7E907061C455184DD7CB2ED1DC6A1, mem on close (mb), free: 250, total: 396, max: 672
2016-05-18 01:33:01,656 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E654A931A7A841E8A650D30AAFDCC4A1
2016-05-18 01:33:01,656 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E654A931A7A841E8A650D30AAFDCC4A1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:01,657 INFO output.MROutput (MROutput.java:close(526)) - A59D28D6A5304243ABA39F425135D08A closed
2016-05-18 01:33:01,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:01 Completed running task attempt: attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,658 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:01,658 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535180903_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:01,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535180903_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:01,660 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535180903_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,661 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:01,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskAttemptId=attempt_1463535180903_0001_1_01_000000_0, creationTime=1463535181595, allocationTime=1463535181597, startTime=1463535181599, finishTime=1463535181661, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:01,661 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:01,662 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,662 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535180903_0001_1_01_000000_0
2016-05-18 01:33:01,662 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:TASK_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, taskId=task_1463535180903_0001_1_01_000000, startTime=1463535181599, finishTime=1463535181662, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463535180903_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:01,664 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:01,664 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1]
2016-05-18 01:33:01,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:VERTEX_FINISHED]: vertexName=CAD7E907061C455184DD7CB2ED1DC6A1, vertexId=vertex_1463535180903_0001_1_01, initRequestedTime=1463535181491, initedTime=1463535181498, startRequestedTime=1463535181508, startedTime=1463535181508, finishTime=1463535181664, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535181645, Process_Duration=11, Process_End_Time=1463535181656, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535181599, firstTasksToStart=[ task_1463535180903_0001_1_01_000000 ], lastTaskFinishTime=1463535181661, lastTasksToFinish=[ task_1463535180903_0001_1_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535180903_0001_1_01_000000 ], longestDurationTasks=[ task_1463535180903_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:01,665 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:01,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535180903_0001_00_000002, stoppedTime=1463535181665, exitStatus=0
2016-05-18 01:33:01,665 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:01,666 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,666 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535180903_0001_1_00 [E654A931A7A841E8A650D30AAFDCC4A1]
2016-05-18 01:33:01,666 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535180903_0001_1
2016-05-18 01:33:01,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535180903_0001_1
2016-05-18 01:33:01,666 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A59D28D6A5304243ABA39F425135D08A for vertex: vertex_1463535180903_0001_1_01 [CAD7E907061C455184DD7CB2ED1DC6A1], outputName: A59D28D6A5304243ABA39F425135D08A
2016-05-18 01:33:01,666 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:01,667 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535180903_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:01,673 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535180903_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:01,674 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535180903_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:01,674 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535180903_0001_00_000002, Exiting
2016-05-18 01:33:01,674 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,674 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535180903_0001_00_000002
2016-05-18 01:33:01,674 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535180903_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:01,674 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535180903_0001_00_000002 completed successfully
2016-05-18 01:33:01,677 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A59D28D6A5304243ABA39F425135D08A of vertex/vertexGroup:CAD7E907061C455184DD7CB2ED1DC6A1 isVertexGroupOutput:false
2016-05-18 01:33:01,678 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:01,680 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535180903_0001_1, queueSize=0
2016-05-18 01:33:01,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535180903_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535180903_0001_1, startTime=1463535181489, finishTime=1463535181678, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=320, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=135, OUTPUT_BYTES_WITH_OVERHEAD=151, OUTPUT_BYTES_PHYSICAL=155, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=155, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=155, SHUFFLE_BYTES_DECOMPRESSED=151, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=155, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070363220, Process_Duration=13, Process_End_Time=2927070363233, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:01,681 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535180903_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:01,681 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535180903_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:01 Completed Dag: dag_1463535180903_0001_1
2016-05-18 01:33:01,681 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535180903_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:01,682 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535180903_0001_1_post
2016-05-18 01:33:01,682 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:01,682 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:01,682 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:01,682 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, with id=dag_1463535180903_0001_1
2016-05-18 01:33:01,682 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1] test/(1/1) ...eSortUnknown/none-unknown, with id=dag_1463535180903_0001_1
2016-05-18 01:33:01,695 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...eSortUnknown/none-unknown, applicationId=application_1463535180903_0001
2016-05-18 01:33:01,697 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:01,697 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:01,698 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:01,698 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:01,698 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:01,698 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:01,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:01,699 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:01,699 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:01,699 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:01,699 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:01,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:01,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:01,702 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:01,705 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/DA2AB4A0761F4CA5A0866C60DD32566E/E9960377B76747DDA59B8ED39A72C8C1/.tez/application_1463535180903_0001
2016-05-18 01:33:01,705 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown/_temporary
2016-05-18 01:33:01,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.833
2016-05-18 01:33:01,748 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupNoneSortUnknown/none-unknown
2016-05-18 01:33:01,780 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:01,780 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:01,831 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:01,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 01:33:01,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:01,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:01,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:01,867 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:33:01,868 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre
2016-05-18 01:33:01,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:01,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:01,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"]
2016-05-18 01:33:01,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:01,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:01,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:01,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:01,886 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:01,906 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:01,910 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:01,911 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:01,911 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:01,920 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001 doesn't exist and is created
2016-05-18 01:33:01,941 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:01,941 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:01,942 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd
2016-05-18 01:33:01,958 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535181911_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:01,958 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:01,959 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:01,960 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:01,961 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:01,963 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:01,963 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:01,964 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:01,964 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:01,965 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:01,965 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:01,965 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:01,968 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:01,969 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:01,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:01,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir
2016-05-18 01:33:01,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/history.txt.appattempt_1463535181911_0001_000000, maxErrors=10
2016-05-18 01:33:01,970 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:01,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535181911_0001_000000, appSubmitTime=1463535181957, launchTime=1463535181958
2016-05-18 01:33:01,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:01,987 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34625
2016-05-18 01:33:01,987 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535181911
2016-05-18 01:33:01,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535181911_0001_000000, startTime=1463535181989
2016-05-18 01:33:01,990 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,041 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,041 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,042 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001, dagName=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,068 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,068 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:02,069 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535181911_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/dag_1463535181911_0001_1.dot
2016-05-18 01:33:02,070 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001_wd/localmode-log-dir/dag_1463535181911_0001_1-tez-dag.pb.txt
2016-05-18 01:33:02,075 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535181911_0001_1
2016-05-18 01:33:02,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,076 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02 Running Dag: dag_1463535181911_0001_1
2016-05-18 01:33:02,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535181911_0001_1, submitTime=1463535182068
2016-05-18 01:33:02,092 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001/tez-conf.pb" } size: 40938 timestamp: 1463535181000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:02,092 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,092 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,092 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:02,093 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:02,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535181911_0001_1, initTime=1463535182093
2016-05-18 01:33:02,094 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:02,094 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:02,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_STARTED]: dagID=dag_1463535181911_0001_1, startTime=1463535182095
2016-05-18 01:33:02,095 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:02,095 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F9FB7ED0624B47BC882DF714B6949DC1 : {1FE6BD0DC88D434898269C5A9D551BA7={InputName=1FE6BD0DC88D434898269C5A9D551BA7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:02,096 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1FE6BD0DC88D434898269C5A9D551BA7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:02,096 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,096 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,097 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001, dagName=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,097 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:02,097 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1FE6BD0DC88D434898269C5A9D551BA7 on vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,101 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1FE6BD0DC88D434898269C5A9D551BA7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:02,104 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:02,104 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:33:02,105 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:02,105 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:02,105 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1FE6BD0DC88D434898269C5A9D551BA7 on vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,106 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,107 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1FE6BD0DC88D434898269C5A9D551BA7=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:02,108 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] parallelism set to 1
2016-05-18 01:33:02,108 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,108 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], numAdditionalOutputs=1
2016-05-18 01:33:02,108 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=083C7AB3509047089BD4A6838B62AC11, vertex=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:02,111 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F9FB7ED0624B47BC882DF714B6949DC1:083C7AB3509047089BD4A6838B62AC11 using oldmapred API
2016-05-18 01:33:02,112 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=083C7AB3509047089BD4A6838B62AC11, vertexName=F9FB7ED0624B47BC882DF714B6949DC1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:02,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, initRequestedTime=1463535182095, initedTime=1463535182108, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:02,115 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] with distanceFromRoot: 0
2016-05-18 01:33:02,115 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:02,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535181911_0001_1_00, startRequestedTime=1463535182097, startedTime=1463535182115
2016-05-18 01:33:02,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535181911_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:02,117 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:02,117 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F9FB7ED0624B47BC882DF714B6949DC1
2016-05-18 01:33:02,117 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_STARTED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskId=task_1463535181911_0001_1_00_000000, scheduledTime=1463535182118, launchTime=1463535182118
2016-05-18 01:33:02,119 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,120 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:02,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535181911_0001_00_000001, launchTime=1463535182122
2016-05-18 01:33:02,123 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,124 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535181911_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,124 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535181911_0001_1_00_000000_0] started. Is using containerId: [container_1463535181911_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0, startTime=1463535182124, containerId=container_1463535181911_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:02,125 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,125 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,125 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:02,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, VertexName: F9FB7ED0624B47BC882DF714B6949DC1, VertexParallelism: 1, TaskAttemptID:attempt_1463535181911_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1FE6BD0DC88D434898269C5A9D551BA7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=083C7AB3509047089BD4A6838B62AC11, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:02,125 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:02,126 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535181911_0001, with dag id: dag_1463535181911_0001_1
2016-05-18 01:33:02,133 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535181911_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:02,133 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535181911_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:02,133 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:02,137 INFO input.MRInput (MRInput.java:initialize(440)) - 1FE6BD0DC88D434898269C5A9D551BA7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:02,138 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,146 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, ordinal: 0
2016-05-18 01:33:02,146 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, mem on start (mb), free: 173, total: 396, max: 672
2016-05-18 01:33:02,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:02,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:02,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:02,150 INFO output.MROutput (MROutput.java:initialize(421)) - 083C7AB3509047089BD4A6838B62AC11: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:02,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:02,150 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1FE6BD0DC88D434898269C5A9D551BA7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [083C7AB3509047089BD4A6838B62AC11:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:02,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:02,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:02,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:02,151 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1FE6BD0DC88D434898269C5A9D551BA7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [083C7AB3509047089BD4A6838B62AC11:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:02,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:02,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1FE6BD0DC88D434898269C5A9D551BA7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:02,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:02,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:02,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:02,154 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0
2016-05-18 01:33:02,157 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:33:02,157 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1FE6BD0DC88D434898269C5A9D551BA7 initialized RecordReader from event
2016-05-18 01:33:02,158 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:02,158 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 1FE6BD0DC88D434898269C5A9D551BA7
2016-05-18 01:33:02,158 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"], id: 083C7AB3509047089BD4A6838B62AC11
2016-05-18 01:33:02,159 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre"]
2016-05-18 01:33:02,159 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:02,159 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:02,161 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught OutOfMemoryException, will not trap, rethrowing
java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:02,163 INFO output.MROutput (MROutput.java:close(526)) - 083C7AB3509047089BD4A6838B62AC11 closed
2016-05-18 01:33:02,164 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535181911_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:02,166 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535181911_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:02,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535181911_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463535182159, Process_Duration=6, Process_End_Time=1463535182165, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-18 01:33:02,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 01:33:02,167 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FB7ED0624B47BC882DF714B6949DC1, mem on close (mb), free: 171, total: 396, max: 672
2016-05-18 01:33:02,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=F9FB7ED0624B47BC882DF714B6949DC1, index=0, interruptedStatus=false
2016-05-18 01:33:02,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=F9FB7ED0624B47BC882DF714B6949DC1, sourceVertex=1FE6BD0DC88D434898269C5A9D551BA7, interruptedStatus=true
2016-05-18 01:33:02,168 INFO output.MROutput (MROutput.java:close(526)) - 083C7AB3509047089BD4A6838B62AC11 closed
2016-05-18 01:33:02,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=F9FB7ED0624B47BC882DF714B6949DC1, sourceVertex=083C7AB3509047089BD4A6838B62AC11, interruptedStatus=true
2016-05-18 01:33:02,172 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535181911_0001_1_00_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:02,173 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:02,173 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,173 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535181911_0001_00_000001
2016-05-18 01:33:02,174 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535181911_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=java.lang.OutOfMemoryError: fake error, errorMessage='TaskExecutionFailure: fake error'}
2016-05-18 01:33:02,174 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463535181911_0001_00_000001 completed but with errors
2016-05-18 01:33:02,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskAttemptId=attempt_1463535181911_0001_1_00_000000_0, creationTime=1463535182119, allocationTime=1463535182121, startTime=1463535182124, finishTime=1463535182172, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535182159, Process_Duration=6, Process_End_Time=1463535182165, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-18 01:33:02,178 INFO container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463535181911_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: fake error
2016-05-18 01:33:02,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535181911_0001_00_000001, stoppedTime=1463535182178, exitStatus=1
2016-05-18 01:33:02,178 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463535181911_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 01:33:02,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:TASK_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, taskId=task_1463535181911_0001_1_00_000000, startTime=1463535182124, finishTime=1463535182179, timeTaken=55, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-18 01:33:02,181 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] because task failed: task_1463535181911_0001_1_00_000000
2016-05-18 01:33:02,182 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] due to trigger: OWN_TASK_FAILURE
2016-05-18 01:33:02,183 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 01:33:02,183 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 01:33:02,183 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,184 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=083C7AB3509047089BD4A6838B62AC11, vertexId=vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,188 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:VERTEX_FINISHED]: vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, initRequestedTime=1463535182095, initedTime=1463535182108, startRequestedTime=1463535182097, startedTime=1463535182115, finishTime=1463535182187, timeTaken=72, status=FAILED, diagnostics=Task failed, taskId=task_1463535181911_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463535182124, firstTasksToStart=[ task_1463535181911_0001_1_00_000000 ], lastTaskFinishTime=1463535182172, lastTasksToFinish=[ task_1463535181911_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 01:33:02,189 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 01:33:02,189 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463535181911_0001_1_00_000000_0 failed: true container: container_1463535181911_0001_00_000001 numFailedTAs: 0
2016-05-18 01:33:02,191 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-18 01:33:02,191 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 01:33:02,191 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 01:33:02,192 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535181911_0001_1, queueSize=0
2016-05-18 01:33:02,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535181911_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535181911_0001_1, startTime=1463535182094, finishTime=1463535182192, timeTaken=98, status=FAILED, diagnostics=Vertex failed, vertexName=F9FB7ED0624B47BC882DF714B6949DC1, vertexId=vertex_1463535181911_0001_1_00, diagnostics=[Task failed, taskId=task_1463535181911_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535181911_0001_1_00_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$1.throwIntentionalException(RegressionPipesPlatformTest.java:583)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0
2016-05-18 01:33:02,193 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463535181911_0001_1_00 [F9FB7ED0624B47BC882DF714B6949DC1]
2016-05-18 01:33:02,193 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535181911_0001_1 finished with state: FAILED
2016-05-18 01:33:02,193 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535181911_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:02 Completed Dag: dag_1463535181911_0001_1
2016-05-18 01:33:02,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535181911_0001_1, dagState=FAILED
2016-05-18 01:33:02,194 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535181911_0001_1_post
2016-05-18 01:33:02,194 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:02,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:02,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, with id=dag_1463535181911_0001_1
2016-05-18 01:33:02,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405] test/(1/1) .../testOOMEPreGroup/oomepre, with id=dag_1463535181911_0001_1
2016-05-18 01:33:02,206 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) .../testOOMEPreGroup/oomepre, applicationId=application_1463535181911_0001
2016-05-18 01:33:02,210 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:02,210 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:02,210 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:02,210 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:02,210 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:02,211 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:02,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:02,212 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:02,212 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:02,212 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,213 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:02,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:02,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,214 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:02,218 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD100136A4241DFBC9E69702749DAC8/FD6394A9801C4B50A720983DD06E5405/.tez/application_1463535181911_0001
2016-05-18 01:33:02,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-18 01:33:02,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-18 01:33:02,220 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre/_temporary
2016-05-18 01:33:02,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.336
exception = cascading.flow.FlowException: local step failed: (1/1) .../testOOMEPreGroup/oomepre
2016-05-18 01:33:02,221 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testOOMEPreGroup/oomepre
2016-05-18 01:33:02,262 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:02,262 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:02,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 01:33:02,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:02,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:02,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:02,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy
2016-05-18 01:33:02,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:02,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"]
2016-05-18 01:33:02,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:02,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:02,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:02,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,343 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:02,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:02,367 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,367 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:02,367 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:02,376 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001 doesn't exist and is created
2016-05-18 01:33:02,396 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:02,396 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:02,397 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd
2016-05-18 01:33:02,413 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535182367_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,414 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,414 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,416 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:02,416 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:02,418 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:02,420 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:02,420 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:02,420 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:02,421 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:02,421 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:02,421 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:02,421 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:02,422 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:02,422 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,422 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,422 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:02,423 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:02,425 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:02,425 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:02,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:02,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir
2016-05-18 01:33:02,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/history.txt.appattempt_1463535182367_0001_000000, maxErrors=10
2016-05-18 01:33:02,427 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:02,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535182367_0001_000000, appSubmitTime=1463535182413, launchTime=1463535182413
2016-05-18 01:33:02,433 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60993
2016-05-18 01:33:02,437 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535182367
2016-05-18 01:33:02,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:02,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535182367_0001_000000, startTime=1463535182438
2016-05-18 01:33:02,439 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,496 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,497 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,497 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001, dagName=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,516 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,516 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:02,517 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535182367_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/dag_1463535182367_0001_1.dot
2016-05-18 01:33:02,518 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001_wd/localmode-log-dir/dag_1463535182367_0001_1-tez-dag.pb.txt
2016-05-18 01:33:02,524 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182367_0001_1
2016-05-18 01:33:02,524 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,524 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02 Running Dag: dag_1463535182367_0001_1
2016-05-18 01:33:02,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535182367_0001_1, submitTime=1463535182516
2016-05-18 01:33:02,541 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001/tez-conf.pb" } size: 40938 timestamp: 1463535182000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:02,541 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,542 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,542 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:02,542 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:02,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535182367_0001_1, initTime=1463535182542
2016-05-18 01:33:02,544 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:02,544 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:02,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_STARTED]: dagID=dag_1463535182367_0001_1, startTime=1463535182544
2016-05-18 01:33:02,545 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001, dagName=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy
2016-05-18 01:33:02,545 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:02,545 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BDFF45C31B1C4FC48B7F47203B906A87 : {E81CD38A86FB4C398F525C74135279CC={InputName=E81CD38A86FB4C398F525C74135279CC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:02,545 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E81CD38A86FB4C398F525C74135279CC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:02,545 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,546 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:02,547 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E81CD38A86FB4C398F525C74135279CC on vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,549 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E81CD38A86FB4C398F525C74135279CC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:02,555 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:02,555 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:02,555 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:02,555 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:02,555 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E81CD38A86FB4C398F525C74135279CC on vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,557 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,557 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E81CD38A86FB4C398F525C74135279CC=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:02,558 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] parallelism set to 1
2016-05-18 01:33:02,560 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:02,560 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], numAdditionalOutputs=1
2016-05-18 01:33:02,560 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=1A09E352C89E442FAC701DD3B4FB481A, vertex=vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:02,563 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BDFF45C31B1C4FC48B7F47203B906A87:1A09E352C89E442FAC701DD3B4FB481A using oldmapred API
2016-05-18 01:33:02,563 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=1A09E352C89E442FAC701DD3B4FB481A, vertexName=BDFF45C31B1C4FC48B7F47203B906A87, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:02,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, vertexId=vertex_1463535182367_0001_1_00, initRequestedTime=1463535182545, initedTime=1463535182560, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:02,565 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] with distanceFromRoot: 0
2016-05-18 01:33:02,567 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:02,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535182367_0001_1_00, startRequestedTime=1463535182546, startedTime=1463535182567
2016-05-18 01:33:02,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535182367_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:02,568 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:02,569 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BDFF45C31B1C4FC48B7F47203B906A87
2016-05-18 01:33:02,569 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_STARTED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskId=task_1463535182367_0001_1_00_000000, scheduledTime=1463535182569, launchTime=1463535182569
2016-05-18 01:33:02,570 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,571 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:02,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535182367_0001_00_000001, launchTime=1463535182573
2016-05-18 01:33:02,574 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,575 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182367_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,575 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535182367_0001_1_00_000000_0] started. Is using containerId: [container_1463535182367_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:02,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, startTime=1463535182575, containerId=container_1463535182367_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:02 Starting to run new task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,576 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,577 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535182367_0001, with dag id: dag_1463535182367_0001_1
2016-05-18 01:33:02,577 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:02,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, VertexName: BDFF45C31B1C4FC48B7F47203B906A87, VertexParallelism: 1, TaskAttemptID:attempt_1463535182367_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E81CD38A86FB4C398F525C74135279CC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1A09E352C89E442FAC701DD3B4FB481A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:02,579 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:02,579 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,584 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:02,586 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535182367_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:02,588 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535182367_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:02,591 INFO input.MRInput (MRInput.java:initialize(440)) - E81CD38A86FB4C398F525C74135279CC using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:02,591 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,597 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, ordinal: 0
2016-05-18 01:33:02,597 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, mem on start (mb), free: 312, total: 394, max: 672
2016-05-18 01:33:02,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:02,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:02,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:02,604 INFO output.MROutput (MROutput.java:initialize(421)) - 1A09E352C89E442FAC701DD3B4FB481A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:02,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:02,604 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E81CD38A86FB4C398F525C74135279CC:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [1A09E352C89E442FAC701DD3B4FB481A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:02,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:02,605 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:02,605 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:02,605 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E81CD38A86FB4C398F525C74135279CC:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [1A09E352C89E442FAC701DD3B4FB481A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:02,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:02,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E81CD38A86FB4C398F525C74135279CC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:02,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:02,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:02,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:02,608 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,609 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:02,609 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E81CD38A86FB4C398F525C74135279CC initialized RecordReader from event
2016-05-18 01:33:02,612 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,613 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: E81CD38A86FB4C398F525C74135279CC
2016-05-18 01:33:02,613 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"], id: 1A09E352C89E442FAC701DD3B4FB481A
2016-05-18 01:33:02,613 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy"]
2016-05-18 01:33:02,614 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,614 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:02,616 INFO output.MROutput (MROutput.java:close(526)) - 1A09E352C89E442FAC701DD3B4FB481A closed
2016-05-18 01:33:02,617 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535182367_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:02,618 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,618 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDFF45C31B1C4FC48B7F47203B906A87, mem on close (mb), free: 310, total: 394, max: 672
2016-05-18 01:33:02,621 INFO output.MROutput (MROutput.java:close(526)) - 1A09E352C89E442FAC701DD3B4FB481A closed
2016-05-18 01:33:02,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:02 Completed running task attempt: attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,623 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:02,623 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535182367_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:02,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535182367_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:02,627 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535182367_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:02,627 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:02,627 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:02,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskAttemptId=attempt_1463535182367_0001_1_00_000000_0, creationTime=1463535182570, allocationTime=1463535182572, startTime=1463535182575, finishTime=1463535182627, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:02,628 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,628 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535182367_0001_1_00_000000_0
2016-05-18 01:33:02,628 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:TASK_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, taskId=task_1463535182367_0001_1_00_000000, startTime=1463535182575, finishTime=1463535182628, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463535182367_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:02,629 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:02,629 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87]
2016-05-18 01:33:02,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:VERTEX_FINISHED]: vertexName=BDFF45C31B1C4FC48B7F47203B906A87, vertexId=vertex_1463535182367_0001_1_00, initRequestedTime=1463535182545, initedTime=1463535182560, startRequestedTime=1463535182546, startedTime=1463535182567, finishTime=1463535182630, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535182575, firstTasksToStart=[ task_1463535182367_0001_1_00_000000 ], lastTaskFinishTime=1463535182627, lastTasksToFinish=[ task_1463535182367_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535182367_0001_1_00_000000 ], longestDurationTasks=[ task_1463535182367_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:02,631 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:02,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535182367_0001_00_000001, stoppedTime=1463535182632, exitStatus=0
2016-05-18 01:33:02,632 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:02,632 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:02,633 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535182367_0001_1
2016-05-18 01:33:02,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535182367_0001_1
2016-05-18 01:33:02,633 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 1A09E352C89E442FAC701DD3B4FB481A for vertex: vertex_1463535182367_0001_1_00 [BDFF45C31B1C4FC48B7F47203B906A87], outputName: 1A09E352C89E442FAC701DD3B4FB481A
2016-05-18 01:33:02,635 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:02,635 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535182367_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:02,639 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535182367_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:02,639 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182367_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:02,639 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535182367_0001_00_000001, Exiting
2016-05-18 01:33:02,639 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,639 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535182367_0001_00_000001
2016-05-18 01:33:02,640 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535182367_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:02,641 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535182367_0001_00_000001 completed successfully
2016-05-18 01:33:02,644 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:1A09E352C89E442FAC701DD3B4FB481A of vertex/vertexGroup:BDFF45C31B1C4FC48B7F47203B906A87 isVertexGroupOutput:false
2016-05-18 01:33:02,644 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:02,646 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535182367_0001_1, queueSize=0
2016-05-18 01:33:02,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182367_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535182367_0001_1, startTime=1463535182544, finishTime=1463535182644, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535182614, Process_Duration=4, Process_End_Time=1463535182618, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:02,646 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535182367_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:02,646 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182367_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:02 Completed Dag: dag_1463535182367_0001_1
2016-05-18 01:33:02,647 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535182367_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:02,647 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182367_0001_1_post
2016-05-18 01:33:02,647 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:02,647 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:02,647 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:02,648 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, with id=dag_1463535182367_0001_1
2016-05-18 01:33:02,648 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC] test/(1/1) ...pesplatform/testCopy/copy, with id=dag_1463535182367_0001_1
2016-05-18 01:33:02,654 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testCopy/copy, applicationId=application_1463535182367_0001
2016-05-18 01:33:02,657 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:02,657 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:02,657 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:02,657 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:02,657 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:02,657 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:02,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:02,658 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:02,658 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:02,658 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,658 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:02,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:02,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:02,659 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:02,663 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/2CD1DF5905B242EF831BF1C65E861B95/DB45F15A6B5949D18FDD5F476253FADC/.tez/application_1463535182367_0001
2016-05-18 01:33:02,663 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testCopy/copy/_temporary
2016-05-18 01:33:02,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.323
2016-05-18 01:33:02,702 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testCopy/copy
2016-05-18 01:33:02,735 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:02,735 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:02,782 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 01:33:02,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 01:33:02,796 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,797 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:02,797 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:02,824 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:02,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace
2016-05-18 01:33:02,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:02,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"]
2016-05-18 01:33:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:02,841 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:02,841 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:02,856 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:02,860 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,861 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:02,861 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:02,872 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001 doesn't exist and is created
2016-05-18 01:33:02,893 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:02,893 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:02,894 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd
2016-05-18 01:33:02,910 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535182861_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:02,911 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,911 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:02,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:02,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:02,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:02,914 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:02,915 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:02,916 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:02,917 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:02,917 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:02,917 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:02,917 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:02,918 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:02,918 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:02,918 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:02,918 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,919 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:02,919 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:02,919 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:02,922 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:02,922 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:02,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:02,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir
2016-05-18 01:33:02,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/history.txt.appattempt_1463535182861_0001_000000, maxErrors=10
2016-05-18 01:33:02,924 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:02,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535182861_0001_000000, appSubmitTime=1463535182909, launchTime=1463535182910
2016-05-18 01:33:02,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:02,934 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57199
2016-05-18 01:33:02,935 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535182861
2016-05-18 01:33:02,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535182861_0001_000000, startTime=1463535182937
2016-05-18 01:33:02,937 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:02,993 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:02,994 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:02,994 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001, dagName=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,013 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,014 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:03,015 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535182861_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/dag_1463535182861_0001_1.dot
2016-05-18 01:33:03,015 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001_wd/localmode-log-dir/dag_1463535182861_0001_1-tez-dag.pb.txt
2016-05-18 01:33:03,023 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182861_0001_1
2016-05-18 01:33:03,023 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,023 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03 Running Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535182861_0001_1, submitTime=1463535183013
2016-05-18 01:33:03,039 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001/tez-conf.pb" } size: 40982 timestamp: 1463535182000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:03,040 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,040 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,040 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:03,040 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:03,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535182861_0001_1, initTime=1463535183040
2016-05-18 01:33:03,042 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:03,043 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:03,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_STARTED]: dagID=dag_1463535182861_0001_1, startTime=1463535183043
2016-05-18 01:33:03,044 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:03,043 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001, dagName=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FA8E2E97BCD24234B2E4B22352791A45 : {26388EC893B14C818417AC578F943D16={InputName=26388EC893B14C818417AC578F943D16}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 26388EC893B14C818417AC578F943D16, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,044 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,045 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:03,045 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 26388EC893B14C818417AC578F943D16 on vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,047 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 26388EC893B14C818417AC578F943D16 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:03,049 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:03,049 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:03,050 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:03,050 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:03,050 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 26388EC893B14C818417AC578F943D16 on vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,052 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,052 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {26388EC893B14C818417AC578F943D16=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:03,053 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] parallelism set to 1
2016-05-18 01:33:03,053 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,053 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], numAdditionalOutputs=1
2016-05-18 01:33:03,053 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=9B3EA2D747BC48BCACA9F2B3E249EBC1, vertex=vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:03,056 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FA8E2E97BCD24234B2E4B22352791A45:9B3EA2D747BC48BCACA9F2B3E249EBC1 using oldmapred API
2016-05-18 01:33:03,056 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=9B3EA2D747BC48BCACA9F2B3E249EBC1, vertexName=FA8E2E97BCD24234B2E4B22352791A45, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:03,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, vertexId=vertex_1463535182861_0001_1_00, initRequestedTime=1463535183044, initedTime=1463535183053, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:03,058 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] with distanceFromRoot: 0
2016-05-18 01:33:03,058 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:03,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535182861_0001_1_00, startRequestedTime=1463535183045, startedTime=1463535183058
2016-05-18 01:33:03,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535182861_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:03,059 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:03,059 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FA8E2E97BCD24234B2E4B22352791A45
2016-05-18 01:33:03,059 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_STARTED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskId=task_1463535182861_0001_1_00_000000, scheduledTime=1463535183060, launchTime=1463535183060
2016-05-18 01:33:03,060 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,061 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:03,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535182861_0001_00_000001, launchTime=1463535183063
2016-05-18 01:33:03,064 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535182861_0001_1_00_000000_0] started. Is using containerId: [container_1463535182861_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:03,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, startTime=1463535183065, containerId=container_1463535182861_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:03,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535182861_0001, with dag id: dag_1463535182861_0001_1
2016-05-18 01:33:03,066 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182861_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,067 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,067 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,069 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:03,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, VertexName: FA8E2E97BCD24234B2E4B22352791A45, VertexParallelism: 1, TaskAttemptID:attempt_1463535182861_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=26388EC893B14C818417AC578F943D16, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9B3EA2D747BC48BCACA9F2B3E249EBC1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:03,070 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:03,071 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,074 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535182861_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:03,075 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535182861_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:03,080 INFO input.MRInput (MRInput.java:initialize(440)) - 26388EC893B14C818417AC578F943D16 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:03,080 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,080 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:03,089 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, ordinal: 0
2016-05-18 01:33:03,089 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, mem on start (mb), free: 232, total: 394, max: 672
2016-05-18 01:33:03,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:03,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:03,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:03,094 INFO output.MROutput (MROutput.java:initialize(421)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:03,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:03,094 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[26388EC893B14C818417AC578F943D16:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [9B3EA2D747BC48BCACA9F2B3E249EBC1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:03,094 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:03,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:03,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:03,095 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[26388EC893B14C818417AC578F943D16:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [9B3EA2D747BC48BCACA9F2B3E249EBC1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:03,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:03,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 26388EC893B14C818417AC578F943D16 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:03,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:03,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:03,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:03,096 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,099 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:03,099 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 26388EC893B14C818417AC578F943D16 initialized RecordReader from event
2016-05-18 01:33:03,100 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:03,101 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 26388EC893B14C818417AC578F943D16
2016-05-18 01:33:03,101 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"], id: 9B3EA2D747BC48BCACA9F2B3E249EBC1
2016-05-18 01:33:03,101 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace"]
2016-05-18 01:33:03,102 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:03,102 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:03,104 INFO output.MROutput (MROutput.java:close(526)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1 closed
2016-05-18 01:33:03,105 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535182861_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:03,106 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,106 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA8E2E97BCD24234B2E4B22352791A45, mem on close (mb), free: 230, total: 394, max: 672
2016-05-18 01:33:03,106 INFO output.MROutput (MROutput.java:close(526)) - 9B3EA2D747BC48BCACA9F2B3E249EBC1 closed
2016-05-18 01:33:03,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,108 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:03,108 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535182861_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:03,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535182861_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:03,109 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535182861_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,109 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,110 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:03,110 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,110 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskAttemptId=attempt_1463535182861_0001_1_00_000000_0, creationTime=1463535183060, allocationTime=1463535183062, startTime=1463535183065, finishTime=1463535183110, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:03,111 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535182861_0001_1_00_000000_0
2016-05-18 01:33:03,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:TASK_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, taskId=task_1463535182861_0001_1_00_000000, startTime=1463535183065, finishTime=1463535183112, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463535182861_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:03,113 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:03,113 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45]
2016-05-18 01:33:03,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:VERTEX_FINISHED]: vertexName=FA8E2E97BCD24234B2E4B22352791A45, vertexId=vertex_1463535182861_0001_1_00, initRequestedTime=1463535183044, initedTime=1463535183053, startRequestedTime=1463535183045, startedTime=1463535183058, finishTime=1463535183114, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535183065, firstTasksToStart=[ task_1463535182861_0001_1_00_000000 ], lastTaskFinishTime=1463535183110, lastTasksToFinish=[ task_1463535182861_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535182861_0001_1_00_000000 ], longestDurationTasks=[ task_1463535182861_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:03,115 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:03,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535182861_0001_00_000001, stoppedTime=1463535183115, exitStatus=0
2016-05-18 01:33:03,116 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:03,116 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,116 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535182861_0001_1
2016-05-18 01:33:03,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535182861_0001_1
2016-05-18 01:33:03,117 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:03,117 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 9B3EA2D747BC48BCACA9F2B3E249EBC1 for vertex: vertex_1463535182861_0001_1_00 [FA8E2E97BCD24234B2E4B22352791A45], outputName: 9B3EA2D747BC48BCACA9F2B3E249EBC1
2016-05-18 01:33:03,117 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535182861_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:03,121 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535182861_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:03,121 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535182861_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:03,121 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535182861_0001_00_000001, Exiting
2016-05-18 01:33:03,121 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,121 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535182861_0001_00_000001
2016-05-18 01:33:03,123 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535182861_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:03,123 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535182861_0001_00_000001 completed successfully
2016-05-18 01:33:03,129 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:9B3EA2D747BC48BCACA9F2B3E249EBC1 of vertex/vertexGroup:FA8E2E97BCD24234B2E4B22352791A45 isVertexGroupOutput:false
2016-05-18 01:33:03,129 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,130 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535182861_0001_1, queueSize=0
2016-05-18 01:33:03,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535182861_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535182861_0001_1, startTime=1463535183043, finishTime=1463535183129, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535183102, Process_Duration=4, Process_End_Time=1463535183106, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:03,131 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535182861_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:03,131 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535182861_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:03 Completed Dag: dag_1463535182861_0001_1
2016-05-18 01:33:03,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535182861_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:03,132 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535182861_0001_1_post
2016-05-18 01:33:03,132 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:03,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:03,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, with id=dag_1463535182861_0001_1
2016-05-18 01:33:03,132 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B] test/(1/1) ...wnReplace/unknown-replace, with id=dag_1463535182861_0001_1
2016-05-18 01:33:03,145 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...wnReplace/unknown-replace, applicationId=application_1463535182861_0001
2016-05-18 01:33:03,148 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:03,148 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:03,148 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:03,149 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:03,149 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:03,149 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:03,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:03,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:03,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:03,150 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,151 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:03,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:03,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,153 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:03,156 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3FB4F27CCC3A43DBBB8132562C553E5E/F4AB59CD55D947A9BED34D3E3C949E0B/.tez/application_1463535182861_0001
2016-05-18 01:33:03,157 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace/_temporary
2016-05-18 01:33:03,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.319
2016-05-18 01:33:03,201 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:03,202 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:03,237 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnknownReplace/unknown-replace
2016-05-18 01:33:03,268 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:03,268 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:03,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 01:33:03,329 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 01:33:03,331 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,331 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,331 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:03,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:33:03,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand
2016-05-18 01:33:03,366 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:03,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"]
2016-05-18 01:33:03,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:03,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:03,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:03,370 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...egression/complexlogicand
2016-05-18 01:33:03,370 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:03,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:03,398 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,398 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:03,399 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:03,409 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001 doesn't exist and is created
2016-05-18 01:33:03,429 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:03,429 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:03,430 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd
2016-05-18 01:33:03,446 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535183398_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,446 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,446 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,448 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:03,448 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:03,449 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:03,449 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:03,450 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:03,452 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:03,452 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:03,453 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:03,453 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:03,454 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:03,454 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:03,454 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:03,455 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:03,455 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,455 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,456 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:03,456 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:03,458 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:03,459 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:03,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:03,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir
2016-05-18 01:33:03,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/history.txt.appattempt_1463535183398_0001_000000, maxErrors=10
2016-05-18 01:33:03,461 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:03,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535183398_0001_000000, appSubmitTime=1463535183445, launchTime=1463535183445
2016-05-18 01:33:03,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:03,469 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53363
2016-05-18 01:33:03,470 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535183398
2016-05-18 01:33:03,471 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535183398_0001_000000, startTime=1463535183471
2016-05-18 01:33:03,472 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:03,529 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:03,530 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:03,530 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001, dagName=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,549 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,549 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:03,550 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535183398_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/dag_1463535183398_0001_1.dot
2016-05-18 01:33:03,551 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001_wd/localmode-log-dir/dag_1463535183398_0001_1-tez-dag.pb.txt
2016-05-18 01:33:03,567 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183398_0001_1
2016-05-18 01:33:03,568 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,568 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03 Running Dag: dag_1463535183398_0001_1
2016-05-18 01:33:03,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535183398_0001_1, submitTime=1463535183549
2016-05-18 01:33:03,585 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001/tez-conf.pb" } size: 40938 timestamp: 1463535183000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:03,585 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,585 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,585 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:03,585 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:03,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535183398_0001_1, initTime=1463535183585
2016-05-18 01:33:03,587 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:03,587 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:03,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_STARTED]: dagID=dag_1463535183398_0001_1, startTime=1463535183588
2016-05-18 01:33:03,588 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:03,588 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D89E8741659F4A4AB5D517E5101AB46F : {823A31594A604F9196990374D71566E0={InputName=823A31594A604F9196990374D71566E0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:03,588 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 823A31594A604F9196990374D71566E0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:03,588 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,589 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001, dagName=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand
2016-05-18 01:33:03,590 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:03,590 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 823A31594A604F9196990374D71566E0 on vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,592 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 823A31594A604F9196990374D71566E0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:03,595 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:03,595 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:33:03,596 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:03,596 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:03,596 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 823A31594A604F9196990374D71566E0 on vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,598 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,598 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {823A31594A604F9196990374D71566E0=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:03,599 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] parallelism set to 1
2016-05-18 01:33:03,599 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:03,599 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], numAdditionalOutputs=1
2016-05-18 01:33:03,600 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=555D641D5040473BA1A557F5187EB3DD, vertex=vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:03,602 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D89E8741659F4A4AB5D517E5101AB46F:555D641D5040473BA1A557F5187EB3DD using oldmapred API
2016-05-18 01:33:03,602 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=555D641D5040473BA1A557F5187EB3DD, vertexName=D89E8741659F4A4AB5D517E5101AB46F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:03,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, vertexId=vertex_1463535183398_0001_1_00, initRequestedTime=1463535183588, initedTime=1463535183599, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:03,603 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] with distanceFromRoot: 0
2016-05-18 01:33:03,604 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:03,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535183398_0001_1_00, startRequestedTime=1463535183590, startedTime=1463535183604
2016-05-18 01:33:03,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535183398_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:03,604 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:03,604 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D89E8741659F4A4AB5D517E5101AB46F
2016-05-18 01:33:03,605 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_STARTED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskId=task_1463535183398_0001_1_00_000000, scheduledTime=1463535183605, launchTime=1463535183605
2016-05-18 01:33:03,609 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,610 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:03,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535183398_0001, with dag id: dag_1463535183398_0001_1
2016-05-18 01:33:03,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535183398_0001_00_000001, launchTime=1463535183613
2016-05-18 01:33:03,614 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,615 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535183398_0001_1_00_000000_0] started. Is using containerId: [container_1463535183398_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:03,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, startTime=1463535183615, containerId=container_1463535183398_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:03,618 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183398_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03 Starting to run new task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,619 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,619 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:03,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, VertexName: D89E8741659F4A4AB5D517E5101AB46F, VertexParallelism: 1, TaskAttemptID:attempt_1463535183398_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=823A31594A604F9196990374D71566E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=555D641D5040473BA1A557F5187EB3DD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:03,620 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:03,621 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,621 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535183398_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:03,622 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535183398_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:03,628 INFO input.MRInput (MRInput.java:initialize(440)) - 823A31594A604F9196990374D71566E0 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:03,628 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,630 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:03,641 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, ordinal: 0
2016-05-18 01:33:03,641 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, mem on start (mb), free: 351, total: 395, max: 672
2016-05-18 01:33:03,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:03,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:03,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:03,642 INFO output.MROutput (MROutput.java:initialize(421)) - 555D641D5040473BA1A557F5187EB3DD: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:03,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:03,644 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[823A31594A604F9196990374D71566E0:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [555D641D5040473BA1A557F5187EB3DD:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:03,644 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:03,645 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:03,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:03,646 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[823A31594A604F9196990374D71566E0:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [555D641D5040473BA1A557F5187EB3DD:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:03,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:03,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 823A31594A604F9196990374D71566E0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:03,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:03,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:03,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:03,648 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,650 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:33:03,650 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 823A31594A604F9196990374D71566E0 initialized RecordReader from event
2016-05-18 01:33:03,651 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,652 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 823A31594A604F9196990374D71566E0
2016-05-18 01:33:03,652 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"], id: 555D641D5040473BA1A557F5187EB3DD
2016-05-18 01:33:03,653 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand"]
2016-05-18 01:33:03,653 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,653 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:03,656 INFO output.MROutput (MROutput.java:close(526)) - 555D641D5040473BA1A557F5187EB3DD closed
2016-05-18 01:33:03,656 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535183398_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:03,657 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,658 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D89E8741659F4A4AB5D517E5101AB46F, mem on close (mb), free: 349, total: 395, max: 672
2016-05-18 01:33:03,658 INFO output.MROutput (MROutput.java:close(526)) - 555D641D5040473BA1A557F5187EB3DD closed
2016-05-18 01:33:03,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:03 Completed running task attempt: attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,659 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:03,659 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535183398_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:03,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535183398_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1][cascading.flow.SliceCounters Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=1]]
2016-05-18 01:33:03,661 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535183398_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,661 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:03,661 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:03,661 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,661 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskAttemptId=attempt_1463535183398_0001_1_00_000000_0, creationTime=1463535183609, allocationTime=1463535183612, startTime=1463535183615, finishTime=1463535183661, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:03,662 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535183398_0001_1_00_000000_0
2016-05-18 01:33:03,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:TASK_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, taskId=task_1463535183398_0001_1_00_000000, startTime=1463535183615, finishTime=1463535183662, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463535183398_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1
2016-05-18 01:33:03,663 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:03,664 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F]
2016-05-18 01:33:03,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:VERTEX_FINISHED]: vertexName=D89E8741659F4A4AB5D517E5101AB46F, vertexId=vertex_1463535183398_0001_1_00, initRequestedTime=1463535183588, initedTime=1463535183599, startRequestedTime=1463535183590, startedTime=1463535183604, finishTime=1463535183664, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1, vertexStats=firstTaskStartTime=1463535183615, firstTasksToStart=[ task_1463535183398_0001_1_00_000000 ], lastTaskFinishTime=1463535183661, lastTasksToFinish=[ task_1463535183398_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535183398_0001_1_00_000000 ], longestDurationTasks=[ task_1463535183398_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:03,665 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:03,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535183398_0001_00_000001, stoppedTime=1463535183665, exitStatus=0
2016-05-18 01:33:03,665 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:03,665 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,665 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535183398_0001_1
2016-05-18 01:33:03,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535183398_0001_1
2016-05-18 01:33:03,665 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:03,665 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 555D641D5040473BA1A557F5187EB3DD for vertex: vertex_1463535183398_0001_1_00 [D89E8741659F4A4AB5D517E5101AB46F], outputName: 555D641D5040473BA1A557F5187EB3DD
2016-05-18 01:33:03,666 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535183398_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:03,672 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535183398_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:03,672 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183398_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:03,673 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535183398_0001_00_000001, Exiting
2016-05-18 01:33:03,673 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,673 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535183398_0001_00_000001
2016-05-18 01:33:03,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535183398_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:03,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535183398_0001_00_000001 completed successfully
2016-05-18 01:33:03,677 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:555D641D5040473BA1A557F5187EB3DD of vertex/vertexGroup:D89E8741659F4A4AB5D517E5101AB46F isVertexGroupOutput:false
2016-05-18 01:33:03,677 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:03,678 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535183398_0001_1, queueSize=0
2016-05-18 01:33:03,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183398_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535183398_0001_1, startTime=1463535183587, finishTime=1463535183677, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535183654, Process_Duration=3, Process_End_Time=1463535183657, Read_Duration=2, Tuples_Read=13, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=1
2016-05-18 01:33:03,679 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535183398_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:03,679 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183398_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:03 Completed Dag: dag_1463535183398_0001_1
2016-05-18 01:33:03,679 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535183398_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:03,679 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183398_0001_1_post
2016-05-18 01:33:03,679 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:03,679 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:03,680 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:03,680 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, with id=dag_1463535183398_0001_1
2016-05-18 01:33:03,680 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55] test/(1/1) ...egression/complexlogicand, with id=dag_1463535183398_0001_1
2016-05-18 01:33:03,687 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...egression/complexlogicand, applicationId=application_1463535183398_0001
2016-05-18 01:33:03,689 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:03,689 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:03,689 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:03,690 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:03,690 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:03,690 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:03,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:03,691 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:03,691 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:03,692 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,692 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:03,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:03,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:03,694 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:03,697 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/108D99855E8840FBAFD5BCBB8A0F556F/D3AF3CC85F1348AD90FB4CD8536E7C55/.tez/application_1463535183398_0001
2016-05-18 01:33:03,698 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand/_temporary
2016-05-18 01:33:03,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.332
2016-05-18 01:33:03,737 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicAnd/regression/complexlogicand
2016-05-18 01:33:03,767 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:03,768 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:03,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 01:33:03,826 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 01:33:03,826 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,827 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:03,827 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:03,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt
2016-05-18 01:33:03,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor
2016-05-18 01:33:03,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:03,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:03,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"]
2016-05-18 01:33:03,863 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:03,863 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:03,864 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:03,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...egression/complexlogicxor
2016-05-18 01:33:03,866 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:03,882 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:03,886 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,886 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:03,886 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:03,895 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001 doesn't exist and is created
2016-05-18 01:33:03,918 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:03,918 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:03,919 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd
2016-05-18 01:33:03,935 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535183886_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:03,936 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,936 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:03,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:03,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:03,939 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:03,939 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:03,941 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:03,942 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:03,943 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:03,943 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:03,943 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:03,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:03,944 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:03,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:03,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:03,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:03,945 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:03,946 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:03,948 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:03,949 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:03,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:03,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir
2016-05-18 01:33:03,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/history.txt.appattempt_1463535183886_0001_000000, maxErrors=10
2016-05-18 01:33:03,950 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:03,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535183886_0001_000000, appSubmitTime=1463535183935, launchTime=1463535183935
2016-05-18 01:33:03,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:03,960 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51435
2016-05-18 01:33:03,962 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535183886
2016-05-18 01:33:03,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535183886_0001_000000, startTime=1463535183964
2016-05-18 01:33:03,964 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:04,019 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:04,019 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:04,019 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001, dagName=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,046 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,046 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:04,048 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535183886_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/dag_1463535183886_0001_1.dot
2016-05-18 01:33:04,048 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001_wd/localmode-log-dir/dag_1463535183886_0001_1-tez-dag.pb.txt
2016-05-18 01:33:04,053 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183886_0001_1
2016-05-18 01:33:04,054 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,054 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04 Running Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535183886_0001_1, submitTime=1463535184046
2016-05-18 01:33:04,071 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001/tez-conf.pb" } size: 40938 timestamp: 1463535183000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,071 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,071 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,071 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,071 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:04,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535183886_0001_1, initTime=1463535184072
2016-05-18 01:33:04,073 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:04,074 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:04,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_STARTED]: dagID=dag_1463535183886_0001_1, startTime=1463535184074
2016-05-18 01:33:04,074 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:04,074 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F57C5A953E9E49E2AB619F031E636965 : {8C7E769C0A914B11A4A6BF1CB33FAB0C={InputName=8C7E769C0A914B11A4A6BF1CB33FAB0C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,074 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 8C7E769C0A914B11A4A6BF1CB33FAB0C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,074 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,075 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,076 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,076 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C on vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,076 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001, dagName=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor
2016-05-18 01:33:04,078 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 8C7E769C0A914B11A4A6BF1CB33FAB0C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,081 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,081 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Total length: 52 Original splits: 2
2016-05-18 01:33:04,081 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 52 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 52 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,081 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:04,082 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C on vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,083 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,083 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {8C7E769C0A914B11A4A6BF1CB33FAB0C=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,086 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] parallelism set to 1
2016-05-18 01:33:04,086 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,086 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], numAdditionalOutputs=1
2016-05-18 01:33:04,086 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8C29811579D948F788C751B2257506DF, vertex=vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:04,089 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F57C5A953E9E49E2AB619F031E636965:8C29811579D948F788C751B2257506DF using oldmapred API
2016-05-18 01:33:04,089 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8C29811579D948F788C751B2257506DF, vertexName=F57C5A953E9E49E2AB619F031E636965, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:04,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F57C5A953E9E49E2AB619F031E636965, vertexId=vertex_1463535183886_0001_1_00, initRequestedTime=1463535184074, initedTime=1463535184086, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,090 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] with distanceFromRoot: 0
2016-05-18 01:33:04,090 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535183886_0001_1_00, startRequestedTime=1463535184076, startedTime=1463535184091
2016-05-18 01:33:04,092 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535183886_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,092 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,092 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F57C5A953E9E49E2AB619F031E636965
2016-05-18 01:33:04,092 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_STARTED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskId=task_1463535183886_0001_1_00_000000, scheduledTime=1463535184093, launchTime=1463535184093
2016-05-18 01:33:04,094 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,095 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:04,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535183886_0001, with dag id: dag_1463535183886_0001_1
2016-05-18 01:33:04,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535183886_0001_00_000001, launchTime=1463535184097
2016-05-18 01:33:04,099 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,100 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535183886_0001_1_00_000000_0] started. Is using containerId: [container_1463535183886_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, startTime=1463535184099, containerId=container_1463535183886_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:04,102 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183886_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,102 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,103 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,103 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, VertexName: F57C5A953E9E49E2AB619F031E636965, VertexParallelism: 1, TaskAttemptID:attempt_1463535183886_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C7E769C0A914B11A4A6BF1CB33FAB0C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8C29811579D948F788C751B2257506DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:04,103 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,104 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,104 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535183886_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:04,105 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535183886_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,108 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,112 INFO input.MRInput (MRInput.java:initialize(440)) - 8C7E769C0A914B11A4A6BF1CB33FAB0C using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,112 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,116 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F57C5A953E9E49E2AB619F031E636965, ordinal: 0
2016-05-18 01:33:04,116 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57C5A953E9E49E2AB619F031E636965, mem on start (mb), free: 271, total: 395, max: 672
2016-05-18 01:33:04,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,122 INFO output.MROutput (MROutput.java:initialize(421)) - 8C29811579D948F788C751B2257506DF: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:04,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,123 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8C29811579D948F788C751B2257506DF:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [8C7E769C0A914B11A4A6BF1CB33FAB0C:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 01:33:04,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:04,123 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8C29811579D948F788C751B2257506DF:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [8C7E769C0A914B11A4A6BF1CB33FAB0C:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 01:33:04,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 8C7E769C0A914B11A4A6BF1CB33FAB0C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,125 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,126 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:0+26, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt:26+26], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=52}
2016-05-18 01:33:04,126 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 8C7E769C0A914B11A4A6BF1CB33FAB0C initialized RecordReader from event
2016-05-18 01:33:04,129 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:04,130 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"] streamed: true, id: 8C7E769C0A914B11A4A6BF1CB33FAB0C
2016-05-18 01:33:04,130 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"], id: 8C29811579D948F788C751B2257506DF
2016-05-18 01:33:04,131 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor"]
2016-05-18 01:33:04,131 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lhs.txt"]
2016-05-18 01:33:04,131 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F57C5A953E9E49E2AB619F031E636965, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,134 INFO output.MROutput (MROutput.java:close(526)) - 8C29811579D948F788C751B2257506DF closed
2016-05-18 01:33:04,135 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535183886_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:04,136 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,136 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57C5A953E9E49E2AB619F031E636965, mem on close (mb), free: 269, total: 395, max: 672
2016-05-18 01:33:04,137 INFO output.MROutput (MROutput.java:close(526)) - 8C29811579D948F788C751B2257506DF closed
2016-05-18 01:33:04,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,138 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:04,138 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535183886_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:04,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535183886_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=13, Tuples_Written=3]]
2016-05-18 01:33:04,140 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535183886_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,140 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,140 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:04,140 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskAttemptId=attempt_1463535183886_0001_1_00_000000_0, creationTime=1463535184094, allocationTime=1463535184096, startTime=1463535184099, finishTime=1463535184140, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,141 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535183886_0001_1_00_000000_0
2016-05-18 01:33:04,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:TASK_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, taskId=task_1463535183886_0001_1_00_000000, startTime=1463535184099, finishTime=1463535184143, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463535183886_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3
2016-05-18 01:33:04,144 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,144 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965]
2016-05-18 01:33:04,142 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:VERTEX_FINISHED]: vertexName=F57C5A953E9E49E2AB619F031E636965, vertexId=vertex_1463535183886_0001_1_00, initRequestedTime=1463535184074, initedTime=1463535184086, startRequestedTime=1463535184076, startedTime=1463535184091, finishTime=1463535184144, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3, vertexStats=firstTaskStartTime=1463535184099, firstTasksToStart=[ task_1463535183886_0001_1_00_000000 ], lastTaskFinishTime=1463535184140, lastTasksToFinish=[ task_1463535183886_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535183886_0001_1_00_000000 ], longestDurationTasks=[ task_1463535183886_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,145 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535183886_0001_00_000001, stoppedTime=1463535184145, exitStatus=0
2016-05-18 01:33:04,146 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:04,146 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,146 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535183886_0001_1
2016-05-18 01:33:04,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535183886_0001_1
2016-05-18 01:33:04,146 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:04,146 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8C29811579D948F788C751B2257506DF for vertex: vertex_1463535183886_0001_1_00 [F57C5A953E9E49E2AB619F031E636965], outputName: 8C29811579D948F788C751B2257506DF
2016-05-18 01:33:04,147 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535183886_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,155 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535183886_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,155 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535183886_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,155 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535183886_0001_00_000001, Exiting
2016-05-18 01:33:04,155 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,155 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535183886_0001_00_000001
2016-05-18 01:33:04,156 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535183886_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,156 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8C29811579D948F788C751B2257506DF of vertex/vertexGroup:F57C5A953E9E49E2AB619F031E636965 isVertexGroupOutput:false
2016-05-18 01:33:04,156 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,156 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535183886_0001_00_000001 completed successfully
2016-05-18 01:33:04,158 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535183886_0001_1, queueSize=0
2016-05-18 01:33:04,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535183886_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535183886_0001_1, startTime=1463535184074, finishTime=1463535184157, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463535184131, Process_Duration=5, Process_End_Time=1463535184136, Read_Duration=1, Tuples_Read=13, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=13, Tuples_Written=3
2016-05-18 01:33:04,159 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535183886_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:04,159 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535183886_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:04 Completed Dag: dag_1463535183886_0001_1
2016-05-18 01:33:04,159 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535183886_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:04,159 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535183886_0001_1_post
2016-05-18 01:33:04,160 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,160 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:04,160 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:04,160 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, with id=dag_1463535183886_0001_1
2016-05-18 01:33:04,161 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364] test/(1/1) ...egression/complexlogicxor, with id=dag_1463535183886_0001_1
2016-05-18 01:33:04,166 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...egression/complexlogicxor, applicationId=application_1463535183886_0001
2016-05-18 01:33:04,169 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:04,169 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:04,169 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:04,169 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:04,169 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:04,169 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:04,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:04,170 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:04,170 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:04,170 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,171 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:04,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:04,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,171 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:04,175 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EF89AE2A83D44CBABF83CA1F035CD97A/C113EB0221D64F89903001C63A690364/.tez/application_1463535183886_0001
2016-05-18 01:33:04,176 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor/_temporary
2016-05-18 01:33:04,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.314
2016-05-18 01:33:04,214 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testComplexLogicXor/regression/complexlogicxor
2016-05-18 01:33:04,244 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:04,245 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:04,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 01:33:04,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-05-18 01:33:04,315 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:04,315 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:04,315 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:04,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:33:04,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt
2016-05-18 01:33:04,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost
2016-05-18 01:33:04,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-18 01:33:04,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"]
2016-05-18 01:33:04,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-18 01:33:04,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-18 01:33:04,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-18 01:33:04,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,492 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:04,507 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-18 01:33:04,511 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:04,511 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:04,511 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:04,521 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001 doesn't exist and is created
2016-05-18 01:33:04,541 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:04,541 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:04,542 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd
2016-05-18 01:33:04,557 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535184511_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:04,557 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:04,557 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:04,558 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:04,558 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:04,558 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:04,559 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:04,559 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:04,561 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:04,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:04,562 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:04,562 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:04,563 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:04,564 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:04,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:04,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir
2016-05-18 01:33:04,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/history.txt.appattempt_1463535184511_0001_000000, maxErrors=10
2016-05-18 01:33:04,565 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:04,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535184511_0001_000000, appSubmitTime=1463535184556, launchTime=1463535184556
2016-05-18 01:33:04,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:04,576 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56303
2016-05-18 01:33:04,577 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535184511
2016-05-18 01:33:04,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535184511_0001_000000, startTime=1463535184579
2016-05-18 01:33:04,579 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:04,641 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:04,641 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:04,642 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001, dagName=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,662 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,663 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:04,664 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535184511_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/dag_1463535184511_0001_1.dot
2016-05-18 01:33:04,665 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-log-dir/dag_1463535184511_0001_1-tez-dag.pb.txt
2016-05-18 01:33:04,676 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535184511_0001_1
2016-05-18 01:33:04,676 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,676 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04 Running Dag: dag_1463535184511_0001_1
2016-05-18 01:33:04,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535184511_0001_1, submitTime=1463535184663
2016-05-18 01:33:04,695 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,695 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,696 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,697 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,697 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,697 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,698 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001/tez-conf.pb" } size: 40938 timestamp: 1463535184000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:04,698 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,699 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:04,699 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:04,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535184511_0001_1, initTime=1463535184699
2016-05-18 01:33:04,700 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:04,701 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:04,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_STARTED]: dagID=dag_1463535184511_0001_1, startTime=1463535184701
2016-05-18 01:33:04,701 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:04,701 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 : {DCB1B28207D542E481A43879144C97C8={InputName=DCB1B28207D542E481A43879144C97C8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,701 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DCB1B28207D542E481A43879144C97C8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,701 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,701 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,701 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,702 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,703 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E0B459EF7267495B95C50BAEE54197F0 : {79BC5C67842049FA937D12E1717784E6={InputName=79BC5C67842049FA937D12E1717784E6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:04,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 79BC5C67842049FA937D12E1717784E6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:04,703 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DCB1B28207D542E481A43879144C97C8 on vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,703 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,704 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001, dagName=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,704 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,704 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,705 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,706 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:04,706 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,710 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 79BC5C67842049FA937D12E1717784E6 on vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,711 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:04,711 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,712 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:04,714 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], numAdditionalOutputs=1
2016-05-18 01:33:04,714 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=BBA6B75524E64A87A4E580C0A2F9EF1E, vertex=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:04,714 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DCB1B28207D542E481A43879144C97C8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,719 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D952A5E7D9C44D339A00FECCD6FCD420:BBA6B75524E64A87A4E580C0A2F9EF1E using oldmapred API
2016-05-18 01:33:04,721 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=BBA6B75524E64A87A4E580C0A2F9EF1E, vertexName=D952A5E7D9C44D339A00FECCD6FCD420, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:04,722 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 79BC5C67842049FA937D12E1717784E6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:04,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, initRequestedTime=1463535184706, initedTime=1463535184714, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:04,723 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:04,723 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,723 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:04,723 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,724 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:04,724 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:04,724 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:04,724 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:04,725 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:04,725 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 79BC5C67842049FA937D12E1717784E6 on vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,724 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DCB1B28207D542E481A43879144C97C8 on vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,728 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,728 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {79BC5C67842049FA937D12E1717784E6=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,728 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] parallelism set to 1
2016-05-18 01:33:04,728 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, vertexId=vertex_1463535184511_0001_1_00, initRequestedTime=1463535184703, initedTime=1463535184729, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,729 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] with distanceFromRoot: 0
2016-05-18 01:33:04,729 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_00, startRequestedTime=1463535184706, startedTime=1463535184731
2016-05-18 01:33:04,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,731 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,731 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535184511_0001_1_00 for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 1 numSources: 2
2016-05-18 01:33:04,732 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,732 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,732 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DCB1B28207D542E481A43879144C97C8=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:04,732 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] parallelism set to 1
2016-05-18 01:33:04,733 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:04,732 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,732 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 1 numSources: 2
2016-05-18 01:33:04,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, vertexId=vertex_1463535184511_0001_1_01, initRequestedTime=1463535184701, initedTime=1463535184733, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:04,734 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] with distanceFromRoot: 0
2016-05-18 01:33:04,734 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:04,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskId=task_1463535184511_0001_1_00_000000, scheduledTime=1463535184734, launchTime=1463535184734
2016-05-18 01:33:04,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_01, startRequestedTime=1463535184706, startedTime=1463535184735
2016-05-18 01:33:04,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535184511_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:04,736 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,737 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,737 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,737 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,737 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535184511_0001_1_01 for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] numStartedSources: 2 numSources: 2
2016-05-18 01:33:04,738 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] with distanceFromRoot: 1
2016-05-18 01:33:04,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskId=task_1463535184511_0001_1_01_000000, scheduledTime=1463535184738, launchTime=1463535184738
2016-05-18 01:33:04,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535184511_0001_1_02, startRequestedTime=1463535184737, startedTime=1463535184738
2016-05-18 01:33:04,739 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:04,740 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D952A5E7D9C44D339A00FECCD6FCD420 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:04,741 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E0B459EF7267495B95C50BAEE54197F0 in vertex: D952A5E7D9C44D339A00FECCD6FCD420 numBipartiteSourceTasks: 1
2016-05-18 01:33:04,741 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 in vertex: D952A5E7D9C44D339A00FECCD6FCD420 numBipartiteSourceTasks: 2
2016-05-18 01:33:04,741 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,754 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:04,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000001, launchTime=1463535184756
2016-05-18 01:33:04,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000002, launchTime=1463535184757
2016-05-18 01:33:04,757 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,759 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_00_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,759 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, startTime=1463535184758, containerId=container_1463535184511_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:04,760 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,761 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,761 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,762 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,761 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_01_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,762 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, startTime=1463535184761, containerId=container_1463535184511_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:04,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: E0B459EF7267495B95C50BAEE54197F0, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79BC5C67842049FA937D12E1717784E6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:04,762 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463535184511_0001, with dag id: dag_1463535184511_0001_1
2016-05-18 01:33:04,767 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,767 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,767 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,768 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: FF2A0F3C18AB4F6EBCF105A139B26BC5, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DCB1B28207D542E481A43879144C97C8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:04,768 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,769 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535184511_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:04,769 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,773 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,774 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,781 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535184511_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:33:04,781 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:04,785 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,789 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, ordinal: 1
2016-05-18 01:33:04,789 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, mem on start (mb), free: 353, total: 395, max: 672
2016-05-18 01:33:04,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,791 INFO input.MRInput (MRInput.java:initialize(440)) - 79BC5C67842049FA937D12E1717784E6 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,792 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,793 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[79BC5C67842049FA937D12E1717784E6:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D952A5E7D9C44D339A00FECCD6FCD420:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:04,794 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:04,798 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[79BC5C67842049FA937D12E1717784E6:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D952A5E7D9C44D339A00FECCD6FCD420:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:04,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 79BC5C67842049FA937D12E1717784E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,798 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, ordinal: 0
2016-05-18 01:33:04,802 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, mem on start (mb), free: 352, total: 395, max: 672
2016-05-18 01:33:04,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,801 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,798 INFO input.MRInput (MRInput.java:initialize(440)) - DCB1B28207D542E481A43879144C97C8 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:04,803 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:04,804 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,804 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 79BC5C67842049FA937D12E1717784E6 initialized RecordReader from event
2016-05-18 01:33:04,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,805 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DCB1B28207D542E481A43879144C97C8:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D952A5E7D9C44D339A00FECCD6FCD420:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:04,805 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:04,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:04,806 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DCB1B28207D542E481A43879144C97C8:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D952A5E7D9C44D339A00FECCD6FCD420:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:04,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DCB1B28207D542E481A43879144C97C8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:04,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,809 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,811 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,811 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"] streamed: true, id: 79BC5C67842049FA937D12E1717784E6
2016-05-18 01:33:04,811 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']], id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,812 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,813 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,814 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:04,814 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DCB1B28207D542E481A43879144C97C8 initialized RecordReader from event
2016-05-18 01:33:04,815 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D952A5E7D9C44D339A00FECCD6FCD420 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@21fc2741, comparator=cascading.tuple.hadoop.util.TupleComparator@5ba78e1c, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:04,814 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: DCB1B28207D542E481A43879144C97C8
2016-05-18 01:33:04,815 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']], id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,816 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,816 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false
2016-05-18 01:33:04,819 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D952A5E7D9C44D339A00FECCD6FCD420 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5562ff46, comparator=cascading.tuple.hadoop.util.TupleComparator@56657875, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:04,820 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false
2016-05-18 01:33:04,832 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:04,832 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:04,833 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:04,833 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D952A5E7D9C44D339A00FECCD6FCD420: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:04,833 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:33:04,833 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,841 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:04,842 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:04,842 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D952A5E7D9C44D339A00FECCD6FCD420: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:04,842 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D952A5E7D9C44D339A00FECCD6FCD420: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:04,842 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:04,843 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:04,844 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,844 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B459EF7267495B95C50BAEE54197F0, mem on close (mb), free: 149, total: 395, max: 672
2016-05-18 01:33:04,844 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D952A5E7D9C44D339A00FECCD6FCD420: Starting flush of map output
2016-05-18 01:33:04,845 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D952A5E7D9C44D339A00FECCD6FCD420: Span0.length = 5, perItem = 15
2016-05-18 01:33:04,845 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D952A5E7D9C44D339A00FECCD6FCD420: done sorting span=0, length=5, time=0
2016-05-18 01:33:04,845 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D952A5E7D9C44D339A00FECCD6FCD420: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:04,847 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D952A5E7D9C44D339A00FECCD6FCD420: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_00_000000_0_10041_0/file.out
2016-05-18 01:33:04,847 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,848 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A0F3C18AB4F6EBCF105A139B26BC5, mem on close (mb), free: 148, total: 395, max: 672
2016-05-18 01:33:04,848 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D952A5E7D9C44D339A00FECCD6FCD420: Starting flush of map output
2016-05-18 01:33:04,848 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D952A5E7D9C44D339A00FECCD6FCD420: Span0.length = 5, perItem = 15
2016-05-18 01:33:04,849 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D952A5E7D9C44D339A00FECCD6FCD420: done sorting span=0, length=5, time=0
2016-05-18 01:33:04,849 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D952A5E7D9C44D339A00FECCD6FCD420: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:04,850 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D952A5E7D9C44D339A00FECCD6FCD420: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_01_000000_0_10043_0/file.out
2016-05-18 01:33:04,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,854 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:04,855 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:04,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:04,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:04 Completed running task attempt: attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,857 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,857 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:04,858 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,858 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:04,858 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskAttemptId=attempt_1463535184511_0001_1_00_000000_0, creationTime=1463535184735, allocationTime=1463535184755, startTime=1463535184758, finishTime=1463535184857, timeTaken=99, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,858 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,858 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:04,859 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535184511_0001_1_00_000000_0
2016-05-18 01:33:04,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:04,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, taskId=task_1463535184511_0001_1_00_000000, startTime=1463535184758, finishTime=1463535184859, timeTaken=101, status=SUCCEEDED, successfulAttemptID=attempt_1463535184511_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:04,860 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,860 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0]
2016-05-18 01:33:04,862 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=E0B459EF7267495B95C50BAEE54197F0, vertexId=vertex_1463535184511_0001_1_00, initRequestedTime=1463535184703, initedTime=1463535184729, startRequestedTime=1463535184706, startedTime=1463535184731, finishTime=1463535184860, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184834, Process_Duration=10, Process_End_Time=1463535184844, Read_Duration=10, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535184758, firstTasksToStart=[ task_1463535184511_0001_1_00_000000 ], lastTaskFinishTime=1463535184857, lastTasksToFinish=[ task_1463535184511_0001_1_00_000000 ], minTaskDuration=99, maxTaskDuration=99, avgTaskDuration=99.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535184511_0001_1_00_000000 ], longestDurationTasks=[ task_1463535184511_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,863 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:04,863 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,863 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:04,863 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,863 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,863 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] attempt: attempt_1463535184511_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:04,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000001, stoppedTime=1463535184865, exitStatus=0
2016-05-18 01:33:04,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskAttemptId=attempt_1463535184511_0001_1_01_000000_0, creationTime=1463535184738, allocationTime=1463535184755, startTime=1463535184761, finishTime=1463535184865, timeTaken=104, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:04,866 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_00 [E0B459EF7267495B95C50BAEE54197F0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,866 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,867 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535184511_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,867 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535184511_0001_1_01_000000_0
2016-05-18 01:33:04,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, taskId=task_1463535184511_0001_1_01_000000, startTime=1463535184761, finishTime=1463535184867, timeTaken=106, status=SUCCEEDED, successfulAttemptID=attempt_1463535184511_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:04,869 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:04,869 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5]
2016-05-18 01:33:04,869 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535184511_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,870 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,870 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535184511_0001_00_000001, Exiting
2016-05-18 01:33:04,870 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,870 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000001
2016-05-18 01:33:04,871 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,871 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535184511_0001_00_000001 completed successfully
2016-05-18 01:33:04,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, vertexId=vertex_1463535184511_0001_1_01, initRequestedTime=1463535184701, initedTime=1463535184733, startRequestedTime=1463535184706, startedTime=1463535184735, finishTime=1463535184869, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535184843, Process_Duration=4, Process_End_Time=1463535184847, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535184761, firstTasksToStart=[ task_1463535184511_0001_1_01_000000 ], lastTaskFinishTime=1463535184865, lastTasksToFinish=[ task_1463535184511_0001_1_01_000000 ], minTaskDuration=104, maxTaskDuration=104, avgTaskDuration=104.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535184511_0001_1_01_000000 ], longestDurationTasks=[ task_1463535184511_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:04,883 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000002, stoppedTime=1463535184884, exitStatus=0
2016-05-18 01:33:04,884 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] attempt: attempt_1463535184511_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:04,885 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D952A5E7D9C44D339A00FECCD6FCD420
2016-05-18 01:33:04,885 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:04,885 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E0B459EF7267495B95C50BAEE54197F0, destinationVertexName=D952A5E7D9C44D339A00FECCD6FCD420. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:04,885 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,886 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_01 [FF2A0F3C18AB4F6EBCF105A139B26BC5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,886 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:04,886 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535184511_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:04,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_STARTED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskId=task_1463535184511_0001_1_02_000000, scheduledTime=1463535184886, launchTime=1463535184886
2016-05-18 01:33:04,887 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535184511_0001_00_000003, launchTime=1463535184889
2016-05-18 01:33:04,890 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,890 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535184511_0001_1_02_000000_0] started. Is using containerId: [container_1463535184511_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:04,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0, startTime=1463535184890, containerId=container_1463535184511_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:04,891 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04 Starting to run new task attempt: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,892 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,892 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:04,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, VertexName: D952A5E7D9C44D339A00FECCD6FCD420, VertexParallelism: 1, TaskAttemptID:attempt_1463535184511_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF2A0F3C18AB4F6EBCF105A139B26BC5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=E0B459EF7267495B95C50BAEE54197F0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BBA6B75524E64A87A4E580C0A2F9EF1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:04,892 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:04,893 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,895 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:04,896 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:04,897 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535184511_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,898 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535184511_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:04,900 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:04,903 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535184511_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:04,903 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535184511_0001_00_000002: 39 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:04,904 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535184511_0001_00_000002, Exiting
2016-05-18 01:33:04,904 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,904 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000002
2016-05-18 01:33:04,904 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:04,904 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535184511_0001_00_000002 completed successfully
2016-05-18 01:33:04,909 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, ordinal: 2
2016-05-18 01:33:04,909 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, mem on start (mb), free: 358, total: 395, max: 672
2016-05-18 01:33:04,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:04,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:04,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:04,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:04,913 INFO output.MROutput (MROutput.java:initialize(421)) - BBA6B75524E64A87A4E580C0A2F9EF1E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:04,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:04,913 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E0B459EF7267495B95C50BAEE54197F0:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [FF2A0F3C18AB4F6EBCF105A139B26BC5:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [BBA6B75524E64A87A4E580C0A2F9EF1E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:04,913 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:04,914 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:04,914 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:04,914 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E0B459EF7267495B95C50BAEE54197F0:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [FF2A0F3C18AB4F6EBCF105A139B26BC5:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [BBA6B75524E64A87A4E580C0A2F9EF1E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:04,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:04,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FF2A0F3C18AB4F6EBCF105A139B26BC5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,915 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:04,915 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:04,916 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FF2A0F3C18AB4F6EBCF105A139B26BC5 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E0B459EF7267495B95C50BAEE54197F0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:33:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:33:04,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:04,918 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E0B459EF7267495B95C50BAEE54197F0: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:04,918 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E0B459EF7267495B95C50BAEE54197F0: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:04,919 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E0B459EF7267495B95C50BAEE54197F0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:04,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:04,921 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0
2016-05-18 01:33:04,925 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E0B459EF7267495B95C50BAEE54197F0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:04,927 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:04,927 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']]
2016-05-18 01:33:04,927 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] streamed: true, id: 7D8B18F5FD0646E0AF8EA68C65259B96
2016-05-18 01:33:04,928 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"], id: BBA6B75524E64A87A4E580C0A2F9EF1E
2016-05-18 01:33:04,928 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost"]
2016-05-18 01:33:04,928 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96, for 2 inputs
2016-05-18 01:33:04,928 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:'num'] upper:[{1}:'num']] 7D8B18F5FD0646E0AF8EA68C65259B96, for 2 inputs
2016-05-18 01:33:04,929 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:04,930 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:33:04,930 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535184511_0001_1_00_000000_0_10041} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535184930, TimeTaken=3, Rate=0.03 MB/s
2016-05-18 01:33:04,930 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,931 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,931 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535184511_0001_1_01_000000_0_10043} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535184931, TimeTaken=4, Rate=0.02 MB/s
2016-05-18 01:33:04,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E0B459EF7267495B95C50BAEE54197F0, wasInterrupted=false
2016-05-18 01:33:04,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FF2A0F3C18AB4F6EBCF105A139B26BC5, wasInterrupted=false
2016-05-18 01:33:04,933 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_00_000000_0_10041/file.out, len=95, isLocal=true
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001_wd/localmode-local-dir/output/attempt_1463535184511_0001_1_01_000000_0_10043/file.out, len=95, isLocal=true
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:04,935 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:04,935 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:04,935 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,935 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: Shuffle Runner thread complete
2016-05-18 01:33:04,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:04,935 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:04,936 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:04,936 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,936 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E0B459EF7267495B95C50BAEE54197F0: Shuffle Runner thread complete
2016-05-18 01:33:04,936 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, all 2 inputs ready in: 00:00:00.006
2016-05-18 01:33:04,937 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:04,939 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E0B459EF7267495B95C50BAEE54197F0: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:04,942 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught OutOfMemoryException, will not trap, rethrowing
java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-18 01:33:04,946 INFO output.MROutput (MROutput.java:close(526)) - BBA6B75524E64A87A4E580C0A2F9EF1E closed
2016-05-18 01:33:04,947 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535184511_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:04,948 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535184511_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:04,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535184511_0001_1_02_000000_0: Counters: 33 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=8, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535184936, Process_Duration=12, Process_End_Time=1463535184948, Read_Duration=1, Tuples_Read=2]]
2016-05-18 01:33:04,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(817)) - Joining on EventRouter
2016-05-18 01:33:04,950 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D952A5E7D9C44D339A00FECCD6FCD420, mem on close (mb), free: 353, total: 395, max: 672
2016-05-18 01:33:04,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(851)) - Closed processor for vertex=D952A5E7D9C44D339A00FECCD6FCD420, index=2, interruptedStatus=false
2016-05-18 01:33:04,950 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FF2A0F3C18AB4F6EBCF105A139B26BC5
2016-05-18 01:33:04,951 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A0F3C18AB4F6EBCF105A139B26BC5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:04,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=FF2A0F3C18AB4F6EBCF105A139B26BC5, interruptedStatus=true
2016-05-18 01:33:04,951 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E0B459EF7267495B95C50BAEE54197F0
2016-05-18 01:33:04,951 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E0B459EF7267495B95C50BAEE54197F0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:04,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(887)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=E0B459EF7267495B95C50BAEE54197F0, interruptedStatus=true
2016-05-18 01:33:04,952 INFO output.MROutput (MROutput.java:close(526)) - BBA6B75524E64A87A4E580C0A2F9EF1E closed
2016-05-18 01:33:04,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(911)) - Closed input for vertex=D952A5E7D9C44D339A00FECCD6FCD420, sourceVertex=BBA6B75524E64A87A4E580C0A2F9EF1E, interruptedStatus=true
2016-05-18 01:33:04,954 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535184511_0001_1_02_000000_0 : TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:04,954 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=TASK_ERROR, error=java.lang.OutOfMemoryError: fake error, containerShutdownRequested=false}
2016-05-18 01:33:04,954 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,954 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535184511_0001_00_000003
2016-05-18 01:33:04,954 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535184511_0001_00_000003 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=java.lang.OutOfMemoryError: fake error, errorMessage='TaskExecutionFailure: fake error'}
2016-05-18 01:33:04,954 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(294)) - Container: container_1463535184511_0001_00_000003 completed but with errors
2016-05-18 01:33:04,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskAttemptId=attempt_1463535184511_0001_1_02_000000_0, creationTime=1463535184887, allocationTime=1463535184888, startTime=1463535184890, finishTime=1463535184954, timeTaken=64, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 33, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535184936, Process_Duration=12, Process_End_Time=1463535184948, Read_Duration=1, Tuples_Read=2
2016-05-18 01:33:04,955 INFO container.AMContainerImpl (AMContainerImpl.java:transition(704)) - Container container_1463535184511_0001_00_000003 exited with diagnostics set to TaskExecutionFailure: fake error
2016-05-18 01:33:04,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535184511_0001_00_000003, stoppedTime=1463535184957, exitStatus=1
2016-05-18 01:33:04,958 INFO impl.TaskImpl (TaskImpl.java:transition(1170)) - Failing task: task_1463535184511_0001_1_02_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-18 01:33:04,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:TASK_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, taskId=task_1463535184511_0001_1_02_000000, startTime=1463535184890, finishTime=1463535184958, timeTaken=68, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-18 01:33:04,958 INFO impl.VertexImpl (VertexImpl.java:transition(3353)) - Failing vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] because task failed: task_1463535184511_0001_1_02_000000
2016-05-18 01:33:04,959 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(2102)) - Killing tasks in vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] due to trigger: OWN_TASK_FAILURE
2016-05-18 01:33:04,959 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420], tasks=1, failed=1, killed=0, success=0, completed=1, commits=0, err=OWN_TASK_FAILURE
2016-05-18 01:33:04,959 INFO impl.VertexImpl (VertexImpl.java:finishWithTerminationCause(2084)) - Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
2016-05-18 01:33:04,959 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3087)) - Invoking committer abort for vertex, vertexId=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,959 INFO impl.VertexImpl (VertexImpl.java:run(3095)) - Invoking committer abort for output=BBA6B75524E64A87A4E580C0A2F9EF1E, vertexId=vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:VERTEX_FINISHED]: vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, initRequestedTime=1463535184706, initedTime=1463535184714, startRequestedTime=1463535184737, startedTime=1463535184738, finishTime=1463535184960, timeTaken=222, status=FAILED, diagnostics=Task failed, taskId=task_1463535184511_0001_1_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0
Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] killed/failed due to:OWN_TASK_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1463535184890, firstTasksToStart=[ task_1463535184511_0001_1_02_000000 ], lastTaskFinishTime=1463535184954, lastTasksToFinish=[ task_1463535184511_0001_1_02_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-18 01:33:04,961 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-18 01:33:04,962 INFO node.AMNodeImpl (AMNodeImpl.java:transition(298)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463535184511_0001_1_02_000000_0 failed: true container: container_1463535184511_0001_00_000003 numFailedTAs: 0
2016-05-18 01:33:04,962 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] completed., numCompletedVertices=3, numSuccessfulVertices=2, numFailedVertices=1, numKilledVertices=0, numVertices=3
2016-05-18 01:33:04,962 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=2, numFailedVertices=1, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=VERTEX_FAILURE
2016-05-18 01:33:04,962 INFO impl.DAGImpl (DAGImpl.java:finishWithTerminationCause(1350)) - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2016-05-18 01:33:04,965 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535184511_0001_1, queueSize=0
2016-05-18 01:33:04,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535184511_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535184511_0001_1, startTime=1463535184701, finishTime=1463535184965, timeTaken=264, status=FAILED, diagnostics=Vertex failed, vertexName=D952A5E7D9C44D339A00FECCD6FCD420, vertexId=vertex_1463535184511_0001_1_02, diagnostics=[Task failed, taskId=task_1463535184511_0001_1_02_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Encountered an Error while executing task: attempt_1463535184511_0001_1_02_000000_0:java.lang.OutOfMemoryError: fake error
at cascading.RegressionPipesPlatformTest$2.throwIntentionalException(RegressionPipesPlatformTest.java:635)
at cascading.TestFunction.operate(TestFunction.java:75)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:45)
at cascading.flow.stream.duct.OpenDuct.receive(OpenDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezCoGroupGate.reduce(TezCoGroupGate.java:89)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:351)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:71)
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:59)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:59)
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:36)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420] killed/failed due to:OWN_TASK_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=600, AM_GC_TIME_MILLIS=22, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, cascading.flow.SliceCounters, Process_Begin_Time=2927070369677, Process_Duration=14, Process_End_Time=2927070369691, Read_Duration=13, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:33:04,966 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3082)) - Ignoring multiple aborts for vertex: vertex_1463535184511_0001_1_02 [D952A5E7D9C44D339A00FECCD6FCD420]
2016-05-18 01:33:04,966 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535184511_0001_1 finished with state: FAILED
2016-05-18 01:33:04,966 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535184511_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:04 Completed Dag: dag_1463535184511_0001_1
2016-05-18 01:33:04,967 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535184511_0001_1, dagState=FAILED
2016-05-18 01:33:04,968 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535184511_0001_1_post
2016-05-18 01:33:04,968 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:04,968 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:04,968 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:04,969 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, with id=dag_1463535184511_0001_1
2016-05-18 01:33:04,969 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8] lower*upper/(1/1) ...estOOMEPostGroup/oomepost, with id=dag_1463535184511_0001_1
2016-05-18 01:33:04,972 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...estOOMEPostGroup/oomepost, applicationId=application_1463535184511_0001
2016-05-18 01:33:04,976 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:04,976 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:04,976 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:04,977 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:04,977 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:04,977 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:04,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:04,979 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:04,979 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:04,979 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,979 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:04,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:04,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:04,981 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:04,985 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/6A63FAFAD7D64B409862EA3BED3439E4/E7B688B4668743AAA64ACEE6AF4FEEE8/.tez/application_1463535184511_0001
2016-05-18 01:33:04,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopping all jobs
2016-05-18 01:33:04,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopping: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] stopped all jobs
2016-05-18 01:33:04,986 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost/_temporary
2016-05-18 01:33:04,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:00.498
exception = cascading.flow.FlowException: local step failed: (1/1) ...estOOMEPostGroup/oomepost
2016-05-18 01:33:04,987 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testOOMEPostGroup/oomepost
2016-05-18 01:33:05,015 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:05,016 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:05,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:05,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 01:33:05,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:05,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:05,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:05,090 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:33:05,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified
2016-05-18 01:33:05,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:05,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"]
2016-05-18 01:33:05,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:05,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:05,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:05,151 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,152 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:05,165 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:05,170 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,170 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:05,170 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:05,178 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001 doesn't exist and is created
2016-05-18 01:33:05,198 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:05,198 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:05,199 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd
2016-05-18 01:33:05,215 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535185170_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,216 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,216 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:05,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:05,220 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:05,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:05,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:05,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:05,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:05,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:05,223 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:05,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:05,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:05,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,225 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:05,225 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:05,227 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:05,227 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:05,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:05,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir
2016-05-18 01:33:05,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/history.txt.appattempt_1463535185170_0001_000000, maxErrors=10
2016-05-18 01:33:05,228 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:05,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535185170_0001_000000, appSubmitTime=1463535185215, launchTime=1463535185215
2016-05-18 01:33:05,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:05,241 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49105
2016-05-18 01:33:05,241 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535185170
2016-05-18 01:33:05,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535185170_0001_000000, startTime=1463535185243
2016-05-18 01:33:05,243 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:05,299 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:05,299 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:05,299 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001, dagName=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,317 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,317 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:05,318 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535185170_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/dag_1463535185170_0001_1.dot
2016-05-18 01:33:05,318 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-log-dir/dag_1463535185170_0001_1-tez-dag.pb.txt
2016-05-18 01:33:05,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185170_0001_1
2016-05-18 01:33:05,331 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,331 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05 Running Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535185170_0001_1, submitTime=1463535185317
2016-05-18 01:33:05,352 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/tez-conf.pb" } size: 40938 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,352 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,353 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,355 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001/tez-conf.pb" } size: 40938 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,355 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,356 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,356 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:05,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535185170_0001_1, initTime=1463535185356
2016-05-18 01:33:05,358 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:05,358 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:05,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_STARTED]: dagID=dag_1463535185170_0001_1, startTime=1463535185358
2016-05-18 01:33:05,358 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001, dagName=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified
2016-05-18 01:33:05,359 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:05,359 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A1C305A30A0A4645A7EB63A6FAD6C8DE : {C8BFAA7939F444D3BAED37FF9C2C5159={InputName=C8BFAA7939F444D3BAED37FF9C2C5159}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:05,359 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C8BFAA7939F444D3BAED37FF9C2C5159, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:05,359 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,359 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,359 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,360 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,360 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:05,360 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,363 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C8BFAA7939F444D3BAED37FF9C2C5159 on vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,364 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:05,364 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,365 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185170_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:05,366 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], numAdditionalOutputs=1
2016-05-18 01:33:05,366 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=B2509D7F08954CF79D5A0301A1C4F73E, vertex=vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:05,367 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C8BFAA7939F444D3BAED37FF9C2C5159 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:05,370 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D751A2136EF14677A14F40147006203E:B2509D7F08954CF79D5A0301A1C4F73E using oldmapred API
2016-05-18 01:33:05,370 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=B2509D7F08954CF79D5A0301A1C4F73E, vertexName=D751A2136EF14677A14F40147006203E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:05,371 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:05,371 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:33:05,371 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:05,372 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:05,372 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C8BFAA7939F444D3BAED37FF9C2C5159 on vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D751A2136EF14677A14F40147006203E, vertexId=vertex_1463535185170_0001_1_01, initRequestedTime=1463535185360, initedTime=1463535185366, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:05,374 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:05,375 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,375 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C8BFAA7939F444D3BAED37FF9C2C5159=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:05,375 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] parallelism set to 1
2016-05-18 01:33:05,375 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, vertexId=vertex_1463535185170_0001_1_00, initRequestedTime=1463535185359, initedTime=1463535185376, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:05,376 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] with distanceFromRoot: 0
2016-05-18 01:33:05,377 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:05,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185170_0001_1_00, startRequestedTime=1463535185360, startedTime=1463535185377
2016-05-18 01:33:05,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185170_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:05,378 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,379 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,379 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,379 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535185170_0001_1_00 for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] numStartedSources: 1 numSources: 1
2016-05-18 01:33:05,379 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] with distanceFromRoot: 1
2016-05-18 01:33:05,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535185170_0001, with dag id: dag_1463535185170_0001_1
2016-05-18 01:33:05,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185170_0001_1_01, startRequestedTime=1463535185378, startedTime=1463535185380
2016-05-18 01:33:05,384 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,385 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D751A2136EF14677A14F40147006203E with 0 source tasks and 1 pending tasks
2016-05-18 01:33:05,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_STARTED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskId=task_1463535185170_0001_1_00_000000, scheduledTime=1463535185385, launchTime=1463535185385
2016-05-18 01:33:05,385 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,386 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A1C305A30A0A4645A7EB63A6FAD6C8DE in vertex: D751A2136EF14677A14F40147006203E numBipartiteSourceTasks: 1
2016-05-18 01:33:05,388 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:05,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185170_0001_00_000001, launchTime=1463535185391
2016-05-18 01:33:05,391 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,393 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,394 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185170_0001_1_00_000000_0] started. Is using containerId: [container_1463535185170_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, startTime=1463535185394, containerId=container_1463535185170_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,395 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,395 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,395 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, VertexName: A1C305A30A0A4645A7EB63A6FAD6C8DE, VertexParallelism: 1, TaskAttemptID:attempt_1463535185170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8BFAA7939F444D3BAED37FF9C2C5159, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D751A2136EF14677A14F40147006203E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:05,396 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,397 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,397 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535185170_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:05,397 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185170_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:05,401 INFO input.MRInput (MRInput.java:initialize(440)) - C8BFAA7939F444D3BAED37FF9C2C5159 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:05,401 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,402 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,409 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, ordinal: 0
2016-05-18 01:33:05,409 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, mem on start (mb), free: 264, total: 395, max: 672
2016-05-18 01:33:05,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,410 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C8BFAA7939F444D3BAED37FF9C2C5159:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D751A2136EF14677A14F40147006203E:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:05,411 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,411 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,411 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:05,411 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C8BFAA7939F444D3BAED37FF9C2C5159:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D751A2136EF14677A14F40147006203E:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:05,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C8BFAA7939F444D3BAED37FF9C2C5159 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,413 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,415 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:33:05,415 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - C8BFAA7939F444D3BAED37FF9C2C5159 initialized RecordReader from event
2016-05-18 01:33:05,416 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,416 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: C8BFAA7939F444D3BAED37FF9C2C5159
2016-05-18 01:33:05,417 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'insert']], id: C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,417 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'insert']] C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,419 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D751A2136EF14677A14F40147006203E using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@16e73b4f, comparator=cascading.tuple.hadoop.util.TupleComparator@5ca1a3f6, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:05,419 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D751A2136EF14677A14F40147006203E: , UsingHashComparator=false
2016-05-18 01:33:05,431 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:05,432 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:05,432 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D751A2136EF14677A14F40147006203E: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:05,432 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D751A2136EF14677A14F40147006203E: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:05,432 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:33:05,433 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,435 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,435 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1C305A30A0A4645A7EB63A6FAD6C8DE, mem on close (mb), free: 162, total: 395, max: 672
2016-05-18 01:33:05,436 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D751A2136EF14677A14F40147006203E: Starting flush of map output
2016-05-18 01:33:05,436 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D751A2136EF14677A14F40147006203E: Span0.length = 10, perItem = 16
2016-05-18 01:33:05,436 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D751A2136EF14677A14F40147006203E: done sorting span=0, length=10, time=0
2016-05-18 01:33:05,437 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D751A2136EF14677A14F40147006203E: Heap = SpanIterator<0:9> (span=Span[16777216,160]),
2016-05-18 01:33:05,438 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D751A2136EF14677A14F40147006203E: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-local-dir/output/attempt_1463535185170_0001_1_00_000000_0_10050_0/file.out
2016-05-18 01:33:05,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,444 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:05,444 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185170_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:05,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185170_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:33:05,449 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185170_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,449 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskAttemptId=attempt_1463535185170_0001_1_00_000000_0, creationTime=1463535185385, allocationTime=1463535185389, startTime=1463535185394, finishTime=1463535185449, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,449 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:05,450 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,450 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185170_0001_1_00_000000_0
2016-05-18 01:33:05,451 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, taskId=task_1463535185170_0001_1_00_000000, startTime=1463535185394, finishTime=1463535185451, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463535185170_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:33:05,452 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,452 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_FINISHED]: vertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, vertexId=vertex_1463535185170_0001_1_00, initRequestedTime=1463535185359, initedTime=1463535185376, startRequestedTime=1463535185360, startedTime=1463535185377, finishTime=1463535185452, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535185433, Process_Duration=2, Process_End_Time=1463535185435, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463535185394, firstTasksToStart=[ task_1463535185170_0001_1_00_000000 ], lastTaskFinishTime=1463535185449, lastTasksToFinish=[ task_1463535185170_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185170_0001_1_00_000000 ], longestDurationTasks=[ task_1463535185170_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,454 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185170_0001_00_000001, stoppedTime=1463535185454, exitStatus=0
2016-05-18 01:33:05,454 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] attempt: attempt_1463535185170_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:05,454 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:05,454 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D751A2136EF14677A14F40147006203E
2016-05-18 01:33:05,455 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,455 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, destinationVertexName=D751A2136EF14677A14F40147006203E. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:05,455 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,455 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185170_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_STARTED]: vertexName=D751A2136EF14677A14F40147006203E, taskId=task_1463535185170_0001_1_01_000000, scheduledTime=1463535185455, launchTime=1463535185455
2016-05-18 01:33:05,456 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185170_0001_00_000002, launchTime=1463535185457
2016-05-18 01:33:05,458 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,459 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185170_0001_1_01_000000_0] started. Is using containerId: [container_1463535185170_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D751A2136EF14677A14F40147006203E, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, startTime=1463535185459, containerId=container_1463535185170_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:05,459 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,460 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,461 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,461 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, VertexName: D751A2136EF14677A14F40147006203E, VertexParallelism: 1, TaskAttemptID:attempt_1463535185170_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1C305A30A0A4645A7EB63A6FAD6C8DE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B2509D7F08954CF79D5A0301A1C4F73E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:05,462 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,462 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185170_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,462 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,462 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,463 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185170_0001_00_000001, Exiting
2016-05-18 01:33:05,463 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,463 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185170_0001_00_000001
2016-05-18 01:33:05,466 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535185170_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,466 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185170_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:05,469 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185170_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,469 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:05,469 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185170_0001_00_000001 completed successfully
2016-05-18 01:33:05,472 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,480 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D751A2136EF14677A14F40147006203E, ordinal: 1
2016-05-18 01:33:05,480 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D751A2136EF14677A14F40147006203E, mem on start (mb), free: 157, total: 395, max: 672
2016-05-18 01:33:05,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,482 INFO output.MROutput (MROutput.java:initialize(421)) - B2509D7F08954CF79D5A0301A1C4F73E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:05,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,483 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B2509D7F08954CF79D5A0301A1C4F73E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [A1C305A30A0A4645A7EB63A6FAD6C8DE:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:05,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:05,484 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B2509D7F08954CF79D5A0301A1C4F73E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [A1C305A30A0A4645A7EB63A6FAD6C8DE:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:05,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A1C305A30A0A4645A7EB63A6FAD6C8DE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,485 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:05,486 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:05,487 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A1C305A30A0A4645A7EB63A6FAD6C8DE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:05,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,490 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,492 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:05,495 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:05,495 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:05,496 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535185170_0001_1_00_000000_0_10050} to DISK_DIRECT, csize=57, dsize=53, EndTime=1463535185495, TimeTaken=3, Rate=0.02 MB/s
2016-05-18 01:33:05,496 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,496 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:05,496 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A1C305A30A0A4645A7EB63A6FAD6C8DE, wasInterrupted=false
2016-05-18 01:33:05,497 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:05,497 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'insert']]
2016-05-18 01:33:05,498 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:05,498 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'insert']] streamed: true, id: C7B80D075E404DFAB71C3EFF8D1F2B21
2016-05-18 01:33:05,498 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001_wd/localmode-local-dir/output/attempt_1463535185170_0001_1_00_000000_0_10050/file.out, len=57, isLocal=true
2016-05-18 01:33:05,498 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"], id: B2509D7F08954CF79D5A0301A1C4F73E
2016-05-18 01:33:05,498 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 57 bytes from disk
2016-05-18 01:33:05,499 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:05,499 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:05,499 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:05,499 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 33 bytes
2016-05-18 01:33:05,499 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,499 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: Shuffle Runner thread complete
2016-05-18 01:33:05,501 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified"]
2016-05-18 01:33:05,502 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'insert']] C7B80D075E404DFAB71C3EFF8D1F2B21, for 1 inputs
2016-05-18 01:33:05,502 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D751A2136EF14677A14F40147006203E, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,503 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:05,506 INFO output.MROutput (MROutput.java:close(526)) - B2509D7F08954CF79D5A0301A1C4F73E closed
2016-05-18 01:33:05,507 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535185170_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:05,508 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,508 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D751A2136EF14677A14F40147006203E, mem on close (mb), free: 154, total: 395, max: 672
2016-05-18 01:33:05,508 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A1C305A30A0A4645A7EB63A6FAD6C8DE
2016-05-18 01:33:05,509 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1C305A30A0A4645A7EB63A6FAD6C8DE: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:05,509 INFO output.MROutput (MROutput.java:close(526)) - B2509D7F08954CF79D5A0301A1C4F73E closed
2016-05-18 01:33:05,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,510 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:05,511 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185170_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:05,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185170_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=10]]
2016-05-18 01:33:05,513 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185170_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,513 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, taskAttemptId=attempt_1463535185170_0001_1_01_000000_0, creationTime=1463535185456, allocationTime=1463535185457, startTime=1463535185459, finishTime=1463535185513, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,513 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185170_0001_1_01_000000_0
2016-05-18 01:33:05,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:TASK_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, taskId=task_1463535185170_0001_1_01_000000, startTime=1463535185459, finishTime=1463535185513, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463535185170_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10
2016-05-18 01:33:05,514 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,514 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E]
2016-05-18 01:33:05,513 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:05,515 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,515 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:VERTEX_FINISHED]: vertexName=D751A2136EF14677A14F40147006203E, vertexId=vertex_1463535185170_0001_1_01, initRequestedTime=1463535185360, initedTime=1463535185366, startRequestedTime=1463535185378, startedTime=1463535185380, finishTime=1463535185514, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535185503, Process_Duration=5, Process_End_Time=1463535185508, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535185459, firstTasksToStart=[ task_1463535185170_0001_1_01_000000 ], lastTaskFinishTime=1463535185513, lastTasksToFinish=[ task_1463535185170_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185170_0001_1_01_000000 ], longestDurationTasks=[ task_1463535185170_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,516 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185170_0001_00_000002, stoppedTime=1463535185516, exitStatus=0
2016-05-18 01:33:05,516 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:05,517 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,517 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535185170_0001_1_00 [A1C305A30A0A4645A7EB63A6FAD6C8DE]
2016-05-18 01:33:05,517 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535185170_0001_1
2016-05-18 01:33:05,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535185170_0001_1
2016-05-18 01:33:05,517 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: B2509D7F08954CF79D5A0301A1C4F73E for vertex: vertex_1463535185170_0001_1_01 [D751A2136EF14677A14F40147006203E], outputName: B2509D7F08954CF79D5A0301A1C4F73E
2016-05-18 01:33:05,517 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:05,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185170_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,526 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185170_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,526 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185170_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,526 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185170_0001_00_000002, Exiting
2016-05-18 01:33:05,526 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,526 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185170_0001_00_000002
2016-05-18 01:33:05,527 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185170_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,527 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185170_0001_00_000002 completed successfully
2016-05-18 01:33:05,529 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:B2509D7F08954CF79D5A0301A1C4F73E of vertex/vertexGroup:D751A2136EF14677A14F40147006203E isVertexGroupOutput:false
2016-05-18 01:33:05,529 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,531 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535185170_0001_1, queueSize=0
2016-05-18 01:33:05,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185170_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535185170_0001_1, startTime=1463535185358, finishTime=1463535185529, timeTaken=171, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=340, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=160, OUTPUT_BYTES_WITH_OVERHEAD=53, OUTPUT_BYTES_PHYSICAL=57, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=57, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=57, SHUFFLE_BYTES_DECOMPRESSED=53, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=57, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070370936, Process_Duration=7, Process_End_Time=2927070370943, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:33:05,532 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535185170_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:05,532 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185170_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:05 Completed Dag: dag_1463535185170_0001_1
2016-05-18 01:33:05,532 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535185170_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:05,533 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185170_0001_1_post
2016-05-18 01:33:05,533 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, with id=dag_1463535185170_0001_1
2016-05-18 01:33:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0] test/(1/1) ...Modified/lasteachmodified, with id=dag_1463535185170_0001_1
2016-05-18 01:33:05,541 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...Modified/lasteachmodified, applicationId=application_1463535185170_0001
2016-05-18 01:33:05,543 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:05,543 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:05,543 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:05,543 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:05,543 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:05,544 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:05,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:05,545 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:05,545 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:05,545 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,546 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:05,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:05,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,546 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:05,550 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/599BFFE5B72442838862BB346FCAF90F/B81D5685A80B4A5E804044581643E4F0/.tez/application_1463535185170_0001
2016-05-18 01:33:05,550 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified/_temporary
2016-05-18 01:33:05,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.401
2016-05-18 01:33:05,598 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testLastEachNotModified/lasteachmodified
2016-05-18 01:33:05,627 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:05,627 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:05,671 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:05,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 01:33:05,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:05,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:05,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:05,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:05,698 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown
2016-05-18 01:33:05,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:05,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"]
2016-05-18 01:33:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:05,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:05,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:05,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,711 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:05,725 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:05,729 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,729 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:05,729 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:05,738 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001 doesn't exist and is created
2016-05-18 01:33:05,758 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:05,758 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:05,759 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd
2016-05-18 01:33:05,775 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535185729_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:05,776 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,776 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:05,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:05,778 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:05,779 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:05,781 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:05,781 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:05,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:05,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:05,782 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:05,782 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:05,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:05,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:05,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,783 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:05,784 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:05,784 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:05,786 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:05,787 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:05,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:05,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir
2016-05-18 01:33:05,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/history.txt.appattempt_1463535185729_0001_000000, maxErrors=10
2016-05-18 01:33:05,788 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:05,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535185729_0001_000000, appSubmitTime=1463535185775, launchTime=1463535185775
2016-05-18 01:33:05,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:05,794 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39324
2016-05-18 01:33:05,796 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535185729
2016-05-18 01:33:05,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535185729_0001_000000, startTime=1463535185798
2016-05-18 01:33:05,798 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:05,859 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:05,859 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:05,859 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001, dagName=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,876 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,877 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:05,877 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535185729_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/dag_1463535185729_0001_1.dot
2016-05-18 01:33:05,878 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001_wd/localmode-log-dir/dag_1463535185729_0001_1-tez-dag.pb.txt
2016-05-18 01:33:05,881 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185729_0001_1
2016-05-18 01:33:05,881 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,883 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05 Running Dag: dag_1463535185729_0001_1
2016-05-18 01:33:05,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535185729_0001_1, submitTime=1463535185877
2016-05-18 01:33:05,900 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001/tez-conf.pb" } size: 40982 timestamp: 1463535185000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:05,900 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,900 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,900 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:05,900 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:05,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535185729_0001_1, initTime=1463535185901
2016-05-18 01:33:05,901 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:05,901 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:05,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_STARTED]: dagID=dag_1463535185729_0001_1, startTime=1463535185901
2016-05-18 01:33:05,902 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BB201390967C4EB78762D0F9F35C1925 : {A96177E652AB4CA2BA9300C2196C9610={InputName=A96177E652AB4CA2BA9300C2196C9610}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: A96177E652AB4CA2BA9300C2196C9610, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,902 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,903 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:05,903 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: A96177E652AB4CA2BA9300C2196C9610 on vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,904 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001, dagName=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown
2016-05-18 01:33:05,906 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input A96177E652AB4CA2BA9300C2196C9610 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:05,908 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:05,909 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:05,909 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:05,909 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:05,910 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: A96177E652AB4CA2BA9300C2196C9610 on vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,911 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,911 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {A96177E652AB4CA2BA9300C2196C9610=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:05,912 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] parallelism set to 1
2016-05-18 01:33:05,913 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:05,913 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], numAdditionalOutputs=1
2016-05-18 01:33:05,913 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A4F9C453867E471B91CE1103B468554C, vertex=vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:05,915 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BB201390967C4EB78762D0F9F35C1925:A4F9C453867E471B91CE1103B468554C using oldmapred API
2016-05-18 01:33:05,915 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A4F9C453867E471B91CE1103B468554C, vertexName=BB201390967C4EB78762D0F9F35C1925, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:05,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BB201390967C4EB78762D0F9F35C1925, vertexId=vertex_1463535185729_0001_1_00, initRequestedTime=1463535185902, initedTime=1463535185913, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:05,916 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] with distanceFromRoot: 0
2016-05-18 01:33:05,916 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:05,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535185729_0001_1_00, startRequestedTime=1463535185903, startedTime=1463535185917
2016-05-18 01:33:05,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535185729_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:05,917 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:05,918 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BB201390967C4EB78762D0F9F35C1925
2016-05-18 01:33:05,918 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_STARTED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskId=task_1463535185729_0001_1_00_000000, scheduledTime=1463535185918, launchTime=1463535185918
2016-05-18 01:33:05,919 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,920 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:05,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535185729_0001_00_000001, launchTime=1463535185921
2016-05-18 01:33:05,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535185729_0001, with dag id: dag_1463535185729_0001_1
2016-05-18 01:33:05,923 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,924 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185729_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,924 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535185729_0001_1_00_000000_0] started. Is using containerId: [container_1463535185729_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:05,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, startTime=1463535185924, containerId=container_1463535185729_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:05 Starting to run new task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,925 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,926 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:05,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, VertexName: BB201390967C4EB78762D0F9F35C1925, VertexParallelism: 1, TaskAttemptID:attempt_1463535185729_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A96177E652AB4CA2BA9300C2196C9610, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A4F9C453867E471B91CE1103B468554C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:05,927 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:05,928 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535185729_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:05,928 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535185729_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:05,929 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,932 INFO input.MRInput (MRInput.java:initialize(440)) - A96177E652AB4CA2BA9300C2196C9610 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:05,932 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,933 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:05,943 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BB201390967C4EB78762D0F9F35C1925, ordinal: 0
2016-05-18 01:33:05,943 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB201390967C4EB78762D0F9F35C1925, mem on start (mb), free: 288, total: 394, max: 672
2016-05-18 01:33:05,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:05,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:05,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:05,944 INFO output.MROutput (MROutput.java:initialize(421)) - A4F9C453867E471B91CE1103B468554C: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:05,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:05,945 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A96177E652AB4CA2BA9300C2196C9610:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A4F9C453867E471B91CE1103B468554C:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:05,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:05,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:05,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:05,946 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A96177E652AB4CA2BA9300C2196C9610:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A4F9C453867E471B91CE1103B468554C:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:05,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A96177E652AB4CA2BA9300C2196C9610 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:05,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:05,948 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,949 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:05,949 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - A96177E652AB4CA2BA9300C2196C9610 initialized RecordReader from event
2016-05-18 01:33:05,951 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,952 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: A96177E652AB4CA2BA9300C2196C9610
2016-05-18 01:33:05,952 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"], id: A4F9C453867E471B91CE1103B468554C
2016-05-18 01:33:05,952 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown"]
2016-05-18 01:33:05,953 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:05,954 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BB201390967C4EB78762D0F9F35C1925, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:05,956 INFO output.MROutput (MROutput.java:close(526)) - A4F9C453867E471B91CE1103B468554C closed
2016-05-18 01:33:05,956 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535185729_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:05,957 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,957 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB201390967C4EB78762D0F9F35C1925, mem on close (mb), free: 286, total: 394, max: 672
2016-05-18 01:33:05,957 INFO output.MROutput (MROutput.java:close(526)) - A4F9C453867E471B91CE1103B468554C closed
2016-05-18 01:33:05,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:05 Completed running task attempt: attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,958 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:05,959 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535185729_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:05,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535185729_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:05,960 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535185729_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,960 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:05,960 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:05,960 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,960 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskAttemptId=attempt_1463535185729_0001_1_00_000000_0, creationTime=1463535185919, allocationTime=1463535185921, startTime=1463535185924, finishTime=1463535185960, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:05,961 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535185729_0001_1_00_000000_0
2016-05-18 01:33:05,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:TASK_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, taskId=task_1463535185729_0001_1_00_000000, startTime=1463535185924, finishTime=1463535185962, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463535185729_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:05,963 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:05,963 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925]
2016-05-18 01:33:05,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:VERTEX_FINISHED]: vertexName=BB201390967C4EB78762D0F9F35C1925, vertexId=vertex_1463535185729_0001_1_00, initRequestedTime=1463535185902, initedTime=1463535185913, startRequestedTime=1463535185903, startedTime=1463535185917, finishTime=1463535185963, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535185924, firstTasksToStart=[ task_1463535185729_0001_1_00_000000 ], lastTaskFinishTime=1463535185960, lastTasksToFinish=[ task_1463535185729_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535185729_0001_1_00_000000 ], longestDurationTasks=[ task_1463535185729_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:05,964 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:05,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535185729_0001_00_000001, stoppedTime=1463535185964, exitStatus=0
2016-05-18 01:33:05,965 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:05,965 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,965 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535185729_0001_1
2016-05-18 01:33:05,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535185729_0001_1
2016-05-18 01:33:05,966 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A4F9C453867E471B91CE1103B468554C for vertex: vertex_1463535185729_0001_1_00 [BB201390967C4EB78762D0F9F35C1925], outputName: A4F9C453867E471B91CE1103B468554C
2016-05-18 01:33:05,968 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:05,968 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535185729_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:05,972 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535185729_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:05,972 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535185729_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:05,972 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535185729_0001_00_000001, Exiting
2016-05-18 01:33:05,972 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,972 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535185729_0001_00_000001
2016-05-18 01:33:05,973 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535185729_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:05,973 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535185729_0001_00_000001 completed successfully
2016-05-18 01:33:05,976 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A4F9C453867E471B91CE1103B468554C of vertex/vertexGroup:BB201390967C4EB78762D0F9F35C1925 isVertexGroupOutput:false
2016-05-18 01:33:05,976 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:05,978 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535185729_0001_1, queueSize=0
2016-05-18 01:33:05,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535185729_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535185729_0001_1, startTime=1463535185901, finishTime=1463535185977, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535185954, Process_Duration=3, Process_End_Time=1463535185957, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:05,978 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535185729_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:05,979 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535185729_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:05 Completed Dag: dag_1463535185729_0001_1
2016-05-18 01:33:05,979 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535185729_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:05,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535185729_0001_1_post
2016-05-18 01:33:05,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:05,979 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:05,979 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:05,980 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, with id=dag_1463535185729_0001_1
2016-05-18 01:33:05,980 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5] test/(1/1) ...tform/testUnknown/unknown, with id=dag_1463535185729_0001_1
2016-05-18 01:33:05,986 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testUnknown/unknown, applicationId=application_1463535185729_0001
2016-05-18 01:33:05,992 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:05,992 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:05,992 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:05,992 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:05,992 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:05,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:05,993 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:05,993 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:05,993 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:05,994 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,994 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:05,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:05,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:05,996 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:06,000 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/FD87FB0804744059B1FF1CCC554ECC3B/A1B1C42300BE4626A70011FED9B1E4E5/.tez/application_1463535185729_0001
2016-05-18 01:33:06,001 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnknown/unknown/_temporary
2016-05-18 01:33:06,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.292
2016-05-18 01:33:06,043 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnknown/unknown
2016-05-18 01:33:06,071 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,071 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,108 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 01:33:06,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:33:06,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:06,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:06,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:06,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt
2016-05-18 01:33:06,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"]
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:06,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:06,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,150 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:06,165 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:06,170 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:06,170 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:06,170 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:06,179 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001 doesn't exist and is created
2016-05-18 01:33:06,200 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:06,200 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:06,201 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd
2016-05-18 01:33:06,216 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535186170_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:06,216 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:06,216 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:06,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:06,218 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:06,219 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:06,221 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:06,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:06,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:06,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:06,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:06,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:06,222 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:06,224 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:06,225 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:06,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:06,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir
2016-05-18 01:33:06,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/history.txt.appattempt_1463535186170_0001_000000, maxErrors=10
2016-05-18 01:33:06,226 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:06,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535186170_0001_000000, appSubmitTime=1463535186215, launchTime=1463535186215
2016-05-18 01:33:06,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:06,239 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36006
2016-05-18 01:33:06,239 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535186170
2016-05-18 01:33:06,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535186170_0001_000000, startTime=1463535186241
2016-05-18 01:33:06,241 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:06,301 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:06,301 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:06,301 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001, dagName=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,324 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,324 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:06,325 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535186170_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/dag_1463535186170_0001_1.dot
2016-05-18 01:33:06,326 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001_wd/localmode-log-dir/dag_1463535186170_0001_1-tez-dag.pb.txt
2016-05-18 01:33:06,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535186170_0001_1
2016-05-18 01:33:06,331 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,331 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06 Running Dag: dag_1463535186170_0001_1
2016-05-18 01:33:06,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535186170_0001_1, submitTime=1463535186324
2016-05-18 01:33:06,348 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001/tez-conf.pb" } size: 40982 timestamp: 1463535186000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:06,348 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,349 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,349 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:06,349 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:06,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535186170_0001_1, initTime=1463535186349
2016-05-18 01:33:06,350 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:06,350 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:06,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_STARTED]: dagID=dag_1463535186170_0001_1, startTime=1463535186351
2016-05-18 01:33:06,351 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:06,351 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A311984AD1EE4E8FB13476FDB7F938DD : {572DB052DBB5491792851F935C3301F1={InputName=572DB052DBB5491792851F935C3301F1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:06,351 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 572DB052DBB5491792851F935C3301F1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:06,351 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,352 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,353 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:06,354 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 572DB052DBB5491792851F935C3301F1 on vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,357 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 572DB052DBB5491792851F935C3301F1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:06,359 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:06,359 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 828 Min splitLength: 52428800 New desired splits: 1 Total length: 828 Original splits: 2
2016-05-18 01:33:06,359 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 828 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 828 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:06,360 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:06,360 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 572DB052DBB5491792851F935C3301F1 on vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,362 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:06,362 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001, dagName=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth
2016-05-18 01:33:06,362 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {572DB052DBB5491792851F935C3301F1=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:06,364 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] parallelism set to 1
2016-05-18 01:33:06,365 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:06,365 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], numAdditionalOutputs=1
2016-05-18 01:33:06,365 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=DFA8E854335A4C1DB3A5A4F7AB304EE3, vertex=vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:06,367 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A311984AD1EE4E8FB13476FDB7F938DD:DFA8E854335A4C1DB3A5A4F7AB304EE3 using oldmapred API
2016-05-18 01:33:06,367 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=DFA8E854335A4C1DB3A5A4F7AB304EE3, vertexName=A311984AD1EE4E8FB13476FDB7F938DD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:06,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, vertexId=vertex_1463535186170_0001_1_00, initRequestedTime=1463535186351, initedTime=1463535186365, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:06,368 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] with distanceFromRoot: 0
2016-05-18 01:33:06,368 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:06,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535186170_0001_1_00, startRequestedTime=1463535186353, startedTime=1463535186368
2016-05-18 01:33:06,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535186170_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:06,368 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:06,373 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A311984AD1EE4E8FB13476FDB7F938DD
2016-05-18 01:33:06,373 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_STARTED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskId=task_1463535186170_0001_1_00_000000, scheduledTime=1463535186373, launchTime=1463535186373
2016-05-18 01:33:06,374 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,380 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:06,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535186170_0001_00_000001, launchTime=1463535186382
2016-05-18 01:33:06,383 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,385 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535186170_0001_1_00_000000_0] started. Is using containerId: [container_1463535186170_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:06,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, startTime=1463535186385, containerId=container_1463535186170_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:06,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535186170_0001, with dag id: dag_1463535186170_0001_1
2016-05-18 01:33:06,387 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535186170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06 Starting to run new task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,388 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:06,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, VertexName: A311984AD1EE4E8FB13476FDB7F938DD, VertexParallelism: 1, TaskAttemptID:attempt_1463535186170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=572DB052DBB5491792851F935C3301F1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DFA8E854335A4C1DB3A5A4F7AB304EE3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:06,388 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:06,394 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535186170_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:06,395 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,395 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535186170_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:06,399 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:06,400 INFO input.MRInput (MRInput.java:initialize(440)) - 572DB052DBB5491792851F935C3301F1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:06,400 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,410 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, ordinal: 0
2016-05-18 01:33:06,410 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, mem on start (mb), free: 209, total: 394, max: 672
2016-05-18 01:33:06,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:06,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:06,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:06,418 INFO output.MROutput (MROutput.java:initialize(421)) - DFA8E854335A4C1DB3A5A4F7AB304EE3: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:06,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:06,418 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[572DB052DBB5491792851F935C3301F1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DFA8E854335A4C1DB3A5A4F7AB304EE3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:06,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:06,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:06,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:06,419 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[572DB052DBB5491792851F935C3301F1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DFA8E854335A4C1DB3A5A4F7AB304EE3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:06,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:06,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 572DB052DBB5491792851F935C3301F1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:06,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:06,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:06,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:06,421 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,424 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt:0+414, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt:414+414], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=828}
2016-05-18 01:33:06,425 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 572DB052DBB5491792851F935C3301F1 initialized RecordReader from event
2016-05-18 01:33:06,425 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,426 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"] streamed: true, id: 572DB052DBB5491792851F935C3301F1
2016-05-18 01:33:06,426 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"], id: DFA8E854335A4C1DB3A5A4F7AB304EE3
2016-05-18 01:33:06,426 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth"]
2016-05-18 01:33:06,426 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/critics.txt"]
2016-05-18 01:33:06,427 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:06,429 INFO output.MROutput (MROutput.java:close(526)) - DFA8E854335A4C1DB3A5A4F7AB304EE3 closed
2016-05-18 01:33:06,429 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535186170_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:06,430 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,430 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A311984AD1EE4E8FB13476FDB7F938DD, mem on close (mb), free: 207, total: 394, max: 672
2016-05-18 01:33:06,430 INFO output.MROutput (MROutput.java:close(526)) - DFA8E854335A4C1DB3A5A4F7AB304EE3 closed
2016-05-18 01:33:06,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:06 Completed running task attempt: attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,431 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:06,431 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535186170_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:06,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535186170_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7][cascading.flow.SliceCounters Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=7, Tuples_Written=7]]
2016-05-18 01:33:06,433 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535186170_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:06,433 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:06,433 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:06,433 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskAttemptId=attempt_1463535186170_0001_1_00_000000_0, creationTime=1463535186374, allocationTime=1463535186382, startTime=1463535186385, finishTime=1463535186434, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:06,434 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535186170_0001_1_00_000000_0
2016-05-18 01:33:06,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:TASK_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, taskId=task_1463535186170_0001_1_00_000000, startTime=1463535186385, finishTime=1463535186435, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463535186170_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7
2016-05-18 01:33:06,436 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:06,436 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD]
2016-05-18 01:33:06,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:VERTEX_FINISHED]: vertexName=A311984AD1EE4E8FB13476FDB7F938DD, vertexId=vertex_1463535186170_0001_1_00, initRequestedTime=1463535186351, initedTime=1463535186365, startRequestedTime=1463535186353, startedTime=1463535186368, finishTime=1463535186436, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7, vertexStats=firstTaskStartTime=1463535186385, firstTasksToStart=[ task_1463535186170_0001_1_00_000000 ], lastTaskFinishTime=1463535186434, lastTasksToFinish=[ task_1463535186170_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535186170_0001_1_00_000000 ], longestDurationTasks=[ task_1463535186170_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:06,437 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:06,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535186170_0001_00_000001, stoppedTime=1463535186437, exitStatus=0
2016-05-18 01:33:06,437 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:06,437 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:06,438 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535186170_0001_1
2016-05-18 01:33:06,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535186170_0001_1
2016-05-18 01:33:06,438 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:06,438 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: DFA8E854335A4C1DB3A5A4F7AB304EE3 for vertex: vertex_1463535186170_0001_1_00 [A311984AD1EE4E8FB13476FDB7F938DD], outputName: DFA8E854335A4C1DB3A5A4F7AB304EE3
2016-05-18 01:33:06,438 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535186170_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:06,439 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,439 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535186170_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:06,439 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535186170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:06,439 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535186170_0001_00_000001, Exiting
2016-05-18 01:33:06,440 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,440 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535186170_0001_00_000001
2016-05-18 01:33:06,442 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535186170_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:06,443 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535186170_0001_00_000001 completed successfully
2016-05-18 01:33:06,449 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:DFA8E854335A4C1DB3A5A4F7AB304EE3 of vertex/vertexGroup:A311984AD1EE4E8FB13476FDB7F938DD isVertexGroupOutput:false
2016-05-18 01:33:06,449 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:06,450 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535186170_0001_1, queueSize=0
2016-05-18 01:33:06,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535186170_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535186170_0001_1, startTime=1463535186350, finishTime=1463535186449, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=280, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, cascading.flow.SliceCounters, Process_Begin_Time=1463535186427, Process_Duration=3, Process_End_Time=1463535186430, Read_Duration=0, Tuples_Read=7, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=7, Tuples_Written=7
2016-05-18 01:33:06,451 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535186170_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:06,451 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535186170_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:06 Completed Dag: dag_1463535186170_0001_1
2016-05-18 01:33:06,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535186170_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:06,451 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535186170_0001_1_post
2016-05-18 01:33:06,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:06,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:06,452 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:06,452 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, with id=dag_1463535186170_0001_1
2016-05-18 01:33:06,452 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA] test/(1/1) ...orm/testVarWidth/varwidth, with id=dag_1463535186170_0001_1
2016-05-18 01:33:06,464 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testVarWidth/varwidth, applicationId=application_1463535186170_0001
2016-05-18 01:33:06,466 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:06,466 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:06,466 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:06,466 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:06,467 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:06,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:06,467 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:06,467 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:06,470 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:06,470 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:06,470 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:06,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:06,471 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:06,471 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:06,475 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/87BC93D78DE94E25A4E89BA4506512F6/B6747CF84F784513AA0BF1E6457D32CA/.tez/application_1463535186170_0001
2016-05-18 01:33:06,475 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth/_temporary
2016-05-18 01:33:06,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.326
2016-05-18 01:33:06,520 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testVarWidth/varwidth
2016-05-18 01:33:06,547 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,547 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,575 WARN planner.FlowPlanner (FlowPlanner.java:verifyPipeAssemblyEndPoints(464)) - duplicate head name found, not an error but heads should have unique names: 'test'
2016-05-18 01:33:06,575 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDupeHeadNames/unknown
2016-05-18 01:33:06,611 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,612 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:06,640 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDupeTailNames/unknown
2016-05-18 01:33:06,698 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:06,698 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:06,952 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:06,953 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:07,212 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:07,212 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:07,658 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:07,658 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:08,150 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:08,150 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:08,690 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:08,690 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:09,170 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:09,170 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:09,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:02.174
2016-05-18 01:33:09,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:02.298
2016-05-18 01:33:09,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:09,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:33:09,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:09,418 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt
2016-05-18 01:33:09,569 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] starting
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"]
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] parallel execution of steps is enabled: false
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] executing total steps: 1
2016-05-18 01:33:09,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] allocating management threads: 1
2016-05-18 01:33:09,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] starting step: (1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,731 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [pipeline] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:09,745 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] tez session mode enabled: true
2016-05-18 01:33:09,749 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:09,750 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:09,750 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:09,760 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001 doesn't exist and is created
2016-05-18 01:33:09,780 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:09,780 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:09,781 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd
2016-05-18 01:33:09,798 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535189750_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:09,798 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:09,799 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:09,801 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:09,801 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:09,801 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:09,802 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:09,803 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:09,807 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:09,808 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:09,808 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:09,809 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:09,809 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:09,809 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:09,811 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:09,811 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:09,811 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:09,811 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:09,813 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:09,813 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:09,815 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:09,816 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:09,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:09,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir
2016-05-18 01:33:09,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/history.txt.appattempt_1463535189750_0001_000000, maxErrors=10
2016-05-18 01:33:09,817 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:09,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535189750_0001_000000, appSubmitTime=1463535189797, launchTime=1463535189797
2016-05-18 01:33:09,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:09,827 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48088
2016-05-18 01:33:09,827 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535189750
2016-05-18 01:33:09,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535189750_0001_000000, startTime=1463535189828
2016-05-18 01:33:09,829 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:09,880 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:09,881 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:09,881 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001, dagName=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,898 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,899 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:09,900 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535189750_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/dag_1463535189750_0001_1.dot
2016-05-18 01:33:09,901 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-log-dir/dag_1463535189750_0001_1-tez-dag.pb.txt
2016-05-18 01:33:09,910 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535189750_0001_1
2016-05-18 01:33:09,910 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:09,911 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09 Running Dag: dag_1463535189750_0001_1
2016-05-18 01:33:09,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535189750_0001_1, submitTime=1463535189898
2016-05-18 01:33:09,926 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/tez-conf.pb" } size: 40938 timestamp: 1463535189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:09,927 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,927 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:09,928 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001/tez-conf.pb" } size: 40938 timestamp: 1463535189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:09,928 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,928 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:09,928 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:09,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535189750_0001_1, initTime=1463535189929
2016-05-18 01:33:09,929 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:09,930 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:09,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_STARTED]: dagID=dag_1463535189750_0001_1, startTime=1463535189930
2016-05-18 01:33:09,930 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:09,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B40B2F7CFAB54DCD978B10E5748D3937 : {23A788B655194A789395FF318DE1A1C9={InputName=23A788B655194A789395FF318DE1A1C9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:09,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 23A788B655194A789395FF318DE1A1C9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:09,930 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,931 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,932 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001, dagName=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline
2016-05-18 01:33:09,934 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:09,934 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 23A788B655194A789395FF318DE1A1C9 on vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,934 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,937 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 23A788B655194A789395FF318DE1A1C9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:09,937 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:09,937 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,938 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:09,938 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:09,939 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 32599 Min splitLength: 52428800 New desired splits: 1 Total length: 32599 Original splits: 2
2016-05-18 01:33:09,939 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 32599 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 32599 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:09,939 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:09,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535189750_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:09,940 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], numAdditionalOutputs=1
2016-05-18 01:33:09,940 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F56003740507449D974ACBE04D44D7F9, vertex=vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:09,940 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 23A788B655194A789395FF318DE1A1C9 on vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,943 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FBF8388363404EFBBC6F45982472C1A1:F56003740507449D974ACBE04D44D7F9 using oldmapred API
2016-05-18 01:33:09,943 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F56003740507449D974ACBE04D44D7F9, vertexName=FBF8388363404EFBBC6F45982472C1A1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:09,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, vertexId=vertex_1463535189750_0001_1_01, initRequestedTime=1463535189934, initedTime=1463535189940, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:09,945 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:09,946 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:09,946 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {23A788B655194A789395FF318DE1A1C9=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:09,947 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] parallelism set to 1
2016-05-18 01:33:09,947 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:09,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, vertexId=vertex_1463535189750_0001_1_00, initRequestedTime=1463535189930, initedTime=1463535189947, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:09,948 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] with distanceFromRoot: 0
2016-05-18 01:33:09,948 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:09,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535189750_0001_1_00, startRequestedTime=1463535189934, startedTime=1463535189948
2016-05-18 01:33:09,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535189750_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:09,949 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:09,949 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535189750_0001_1_00 for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] numStartedSources: 1 numSources: 1
2016-05-18 01:33:09,949 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:09,949 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,949 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] with distanceFromRoot: 1
2016-05-18 01:33:09,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535189750_0001_1_01, startRequestedTime=1463535189949, startedTime=1463535189952
2016-05-18 01:33:09,952 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:09,954 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: FBF8388363404EFBBC6F45982472C1A1 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:09,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_STARTED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskId=task_1463535189750_0001_1_00_000000, scheduledTime=1463535189957, launchTime=1463535189957
2016-05-18 01:33:09,958 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:09,958 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B40B2F7CFAB54DCD978B10E5748D3937 in vertex: FBF8388363404EFBBC6F45982472C1A1 numBipartiteSourceTasks: 1
2016-05-18 01:33:09,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] submitted tez dag to app master: application_1463535189750_0001, with dag id: dag_1463535189750_0001_1
2016-05-18 01:33:09,959 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:09,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535189750_0001_00_000001, launchTime=1463535189962
2016-05-18 01:33:09,964 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000001
2016-05-18 01:33:09,966 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535189750_0001_1_00_000000_0] started. Is using containerId: [container_1463535189750_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:09,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, startTime=1463535189965, containerId=container_1463535189750_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:09,966 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09 Starting to run new task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,967 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,967 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:09,968 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:09,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, VertexName: B40B2F7CFAB54DCD978B10E5748D3937, VertexParallelism: 1, TaskAttemptID:attempt_1463535189750_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23A788B655194A789395FF318DE1A1C9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FBF8388363404EFBBC6F45982472C1A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:09,969 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:09,969 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:09,970 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535189750_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:09,971 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535189750_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:09,974 INFO input.MRInput (MRInput.java:initialize(440)) - 23A788B655194A789395FF318DE1A1C9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:09,974 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:09,975 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:09,978 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:09,979 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:10,100 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, ordinal: 0
2016-05-18 01:33:10,100 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, mem on start (mb), free: 290, total: 394, max: 672
2016-05-18 01:33:10,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:10,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:10,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:10,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:10,101 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[23A788B655194A789395FF318DE1A1C9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FBF8388363404EFBBC6F45982472C1A1:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:10,102 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:10,102 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:10,103 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:10,103 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[23A788B655194A789395FF318DE1A1C9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FBF8388363404EFBBC6F45982472C1A1:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:10,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:10,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 23A788B655194A789395FF318DE1A1C9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:10,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:10,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:10,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:10,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:10,105 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,107 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt:0+16299, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt:16299+16300], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=32599}
2016-05-18 01:33:10,107 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 23A788B655194A789395FF318DE1A1C9 initialized RecordReader from event
2016-05-18 01:33:10,108 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:10,137 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"] streamed: true, id: 23A788B655194A789395FF318DE1A1C9
2016-05-18 01:33:10,137 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(pipeline)[by:[{1}:'hash']], id: E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,139 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(pipeline)[by:[{1}:'hash']] E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,140 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - FBF8388363404EFBBC6F45982472C1A1 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5a65f83f, comparator=cascading.tuple.hadoop.util.TupleComparator@5c8d1e76, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:10,141 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for FBF8388363404EFBBC6F45982472C1A1: , UsingHashComparator=false
2016-05-18 01:33:10,155 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:10,155 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:10,155 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for FBF8388363404EFBBC6F45982472C1A1: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:10,155 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - FBF8388363404EFBBC6F45982472C1A1: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:10,236 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 01:33:10,236 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:10,362 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,363 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B40B2F7CFAB54DCD978B10E5748D3937, mem on close (mb), free: 176, total: 394, max: 672
2016-05-18 01:33:10,363 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - FBF8388363404EFBBC6F45982472C1A1: Starting flush of map output
2016-05-18 01:33:10,363 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - FBF8388363404EFBBC6F45982472C1A1: Span0.length = 200, perItem = 176
2016-05-18 01:33:10,385 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - FBF8388363404EFBBC6F45982472C1A1: done sorting span=0, length=200, time=22
2016-05-18 01:33:10,386 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - FBF8388363404EFBBC6F45982472C1A1: Heap = SpanIterator<0:199> (span=Span[16777216,35385]),
2016-05-18 01:33:10,387 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - FBF8388363404EFBBC6F45982472C1A1: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-local-dir/output/attempt_1463535189750_0001_1_00_000000_0_10062_0/file.out
2016-05-18 01:33:10,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,394 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:10,395 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535189750_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:10,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535189750_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27][cascading.flow.StepCounters Tuples_Read=200]]
2016-05-18 01:33:10,398 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535189750_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,398 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,398 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:10,398 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,398 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskAttemptId=attempt_1463535189750_0001_1_00_000000_0, creationTime=1463535189957, allocationTime=1463535189961, startTime=1463535189965, finishTime=1463535190398, timeTaken=433, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:10,399 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535189750_0001_1_00_000000_0
2016-05-18 01:33:10,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, taskId=task_1463535189750_0001_1_00_000000, startTime=1463535189965, finishTime=1463535190399, timeTaken=434, status=SUCCEEDED, successfulAttemptID=attempt_1463535189750_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27, cascading.flow.StepCounters, Tuples_Read=200
2016-05-18 01:33:10,401 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:10,401 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:10,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_FINISHED]: vertexName=B40B2F7CFAB54DCD978B10E5748D3937, vertexId=vertex_1463535189750_0001_1_00, initRequestedTime=1463535189930, initedTime=1463535189947, startRequestedTime=1463535189934, startedTime=1463535189948, finishTime=1463535190401, timeTaken=453, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535190237, Process_Duration=125, Process_End_Time=1463535190362, Read_Duration=3, Tuples_Read=200, Tuples_Written=200, Write_Duration=27, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1463535189965, firstTasksToStart=[ task_1463535189750_0001_1_00_000000 ], lastTaskFinishTime=1463535190398, lastTasksToFinish=[ task_1463535189750_0001_1_00_000000 ], minTaskDuration=433, maxTaskDuration=433, avgTaskDuration=433.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535189750_0001_1_00_000000 ], longestDurationTasks=[ task_1463535189750_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:10,404 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:10,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535189750_0001_00_000001, stoppedTime=1463535190404, exitStatus=0
2016-05-18 01:33:10,404 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] attempt: attempt_1463535189750_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:10,405 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:10,405 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FBF8388363404EFBBC6F45982472C1A1
2016-05-18 01:33:10,405 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,405 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B40B2F7CFAB54DCD978B10E5748D3937, destinationVertexName=FBF8388363404EFBBC6F45982472C1A1. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:10,405 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,405 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535189750_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:10,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_STARTED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskId=task_1463535189750_0001_1_01_000000, scheduledTime=1463535190406, launchTime=1463535190406
2016-05-18 01:33:10,406 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535189750_0001_00_000002, launchTime=1463535190408
2016-05-18 01:33:10,408 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,409 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,409 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535189750_0001_1_01_000000_0] started. Is using containerId: [container_1463535189750_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:10 Starting to run new task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,410 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,410 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,410 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:10,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, VertexName: FBF8388363404EFBBC6F45982472C1A1, VertexParallelism: 1, TaskAttemptID:attempt_1463535189750_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B40B2F7CFAB54DCD978B10E5748D3937, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F56003740507449D974ACBE04D44D7F9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:10,411 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:10,412 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,412 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535189750_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,409 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535189750_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:10,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, startTime=1463535190409, containerId=container_1463535189750_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:10,414 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000001: 16 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:10,415 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:10,415 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535189750_0001_00_000001, Exiting
2016-05-18 01:33:10,415 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,415 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535189750_0001_00_000001
2016-05-18 01:33:10,420 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535189750_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:10,421 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535189750_0001_00_000001 completed successfully
2016-05-18 01:33:10,422 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535189750_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:10,424 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:10,432 INFO output.MROutput (MROutput.java:initialize(421)) - F56003740507449D974ACBE04D44D7F9: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:10,511 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, ordinal: 1
2016-05-18 01:33:10,512 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, mem on start (mb), free: 168, total: 394, max: 672
2016-05-18 01:33:10,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:10,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:10,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:10,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:10,512 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B40B2F7CFAB54DCD978B10E5748D3937:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [F56003740507449D974ACBE04D44D7F9:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:10,512 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:10,513 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:10,513 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:10,513 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B40B2F7CFAB54DCD978B10E5748D3937:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [F56003740507449D974ACBE04D44D7F9:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:10,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:10,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B40B2F7CFAB54DCD978B10E5748D3937 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:10,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:10,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:10,514 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B40B2F7CFAB54DCD978B10E5748D3937: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:10,514 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B40B2F7CFAB54DCD978B10E5748D3937: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:10,516 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B40B2F7CFAB54DCD978B10E5748D3937 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:10,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:10,517 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,518 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B40B2F7CFAB54DCD978B10E5748D3937: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:10,521 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(pipeline)[by:[{1}:'hash']]
2016-05-18 01:33:10,524 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535189750_0001_1_00_000000_0_10062} to DISK_DIRECT, csize=35992, dsize=35988, EndTime=1463535190524, TimeTaken=5, Rate=6.86 MB/s
2016-05-18 01:33:10,524 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,524 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.03 MB/s)
2016-05-18 01:33:10,525 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B40B2F7CFAB54DCD978B10E5748D3937, wasInterrupted=false
2016-05-18 01:33:10,526 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.03 MB/s)
2016-05-18 01:33:10,526 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - B40B2F7CFAB54DCD978B10E5748D3937: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:10,526 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:10,527 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001_wd/localmode-local-dir/output/attempt_1463535189750_0001_1_00_000000_0_10062/file.out, len=35992, isLocal=true
2016-05-18 01:33:10,527 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 35992 bytes from disk
2016-05-18 01:33:10,527 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:10,527 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:10,528 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 35975 bytes
2016-05-18 01:33:10,528 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,528 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B40B2F7CFAB54DCD978B10E5748D3937: Shuffle Runner thread complete
2016-05-18 01:33:10,546 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(pipeline)[by:[{1}:'hash']] streamed: true, id: E6438AF0FDD8483EB8A88D81E849C72A
2016-05-18 01:33:10,546 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"], id: F56003740507449D974ACBE04D44D7F9
2016-05-18 01:33:10,547 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:10,547 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:10,548 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline"]
2016-05-18 01:33:10,549 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(pipeline)[by:[{1}:'hash']] E6438AF0FDD8483EB8A88D81E849C72A, for 1 inputs
2016-05-18 01:33:10,549 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:10,550 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B40B2F7CFAB54DCD978B10E5748D3937: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:10,916 INFO output.MROutput (MROutput.java:close(526)) - F56003740507449D974ACBE04D44D7F9 closed
2016-05-18 01:33:10,917 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535189750_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:33:10,918 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,918 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBF8388363404EFBBC6F45982472C1A1, mem on close (mb), free: 154, total: 394, max: 672
2016-05-18 01:33:10,919 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B40B2F7CFAB54DCD978B10E5748D3937
2016-05-18 01:33:10,919 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B40B2F7CFAB54DCD978B10E5748D3937: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:10,919 INFO output.MROutput (MROutput.java:close(526)) - F56003740507449D974ACBE04D44D7F9 closed
2016-05-18 01:33:10,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:10 Completed running task attempt: attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,921 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:10,922 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535189750_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:10,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535189750_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3][cascading.flow.StepCounters Tuples_Written=200]]
2016-05-18 01:33:10,925 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535189750_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,925 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:10,926 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:10,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,926 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskAttemptId=attempt_1463535189750_0001_1_01_000000_0, creationTime=1463535190406, allocationTime=1463535190407, startTime=1463535190409, finishTime=1463535190926, timeTaken=517, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:10,927 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535189750_0001_1_01_000000_0
2016-05-18 01:33:10,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:TASK_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, taskId=task_1463535189750_0001_1_01_000000, startTime=1463535190409, finishTime=1463535190927, timeTaken=518, status=SUCCEEDED, successfulAttemptID=attempt_1463535189750_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=200
2016-05-18 01:33:10,928 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:10,928 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1]
2016-05-18 01:33:10,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:VERTEX_FINISHED]: vertexName=FBF8388363404EFBBC6F45982472C1A1, vertexId=vertex_1463535189750_0001_1_01, initRequestedTime=1463535189934, initedTime=1463535189940, startRequestedTime=1463535189949, startedTime=1463535189952, finishTime=1463535190928, timeTaken=976, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=200, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=200, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535190549, Process_Duration=369, Process_End_Time=1463535190918, Read_Duration=13, Tuples_Read=200, Tuples_Written=200, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=200, vertexStats=firstTaskStartTime=1463535190409, firstTasksToStart=[ task_1463535189750_0001_1_01_000000 ], lastTaskFinishTime=1463535190926, lastTasksToFinish=[ task_1463535189750_0001_1_01_000000 ], minTaskDuration=517, maxTaskDuration=517, avgTaskDuration=517.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535189750_0001_1_01_000000 ], longestDurationTasks=[ task_1463535189750_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:10,929 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:10,929 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:33:10,929 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,929 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535189750_0001_1_00 [B40B2F7CFAB54DCD978B10E5748D3937]
2016-05-18 01:33:10,929 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535189750_0001_1
2016-05-18 01:33:10,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535189750_0001_1
2016-05-18 01:33:10,930 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:10,930 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F56003740507449D974ACBE04D44D7F9 for vertex: vertex_1463535189750_0001_1_01 [FBF8388363404EFBBC6F45982472C1A1], outputName: F56003740507449D974ACBE04D44D7F9
2016-05-18 01:33:10,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535189750_0001_00_000002, stoppedTime=1463535190930, exitStatus=0
2016-05-18 01:33:10,931 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535189750_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:10,931 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535189750_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:10,933 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535189750_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:10,934 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535189750_0001_00_000002, Exiting
2016-05-18 01:33:10,934 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,934 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535189750_0001_00_000002
2016-05-18 01:33:10,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535189750_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:10,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535189750_0001_00_000002 completed successfully
2016-05-18 01:33:10,944 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F56003740507449D974ACBE04D44D7F9 of vertex/vertexGroup:FBF8388363404EFBBC6F45982472C1A1 isVertexGroupOutput:false
2016-05-18 01:33:10,944 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:33:10,947 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535189750_0001_1, queueSize=0
2016-05-18 01:33:10,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535189750_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535189750_0001_1, startTime=1463535189930, finishTime=1463535190944, timeTaken=1014, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=2130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=200, REDUCE_INPUT_RECORDS=200, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=400, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=400, OUTPUT_BYTES=35385, OUTPUT_BYTES_WITH_OVERHEAD=35988, OUTPUT_BYTES_PHYSICAL=35992, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=35992, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=35992, SHUFFLE_BYTES_DECOMPRESSED=35988, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=35992, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927070380786, Process_Duration=494, Process_End_Time=2927070381280, Read_Duration=16, Tuples_Read=400, Tuples_Written=400, Write_Duration=30, cascading.flow.StepCounters, Tuples_Read=200, Tuples_Written=200
2016-05-18 01:33:10,949 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535189750_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:10,949 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535189750_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:10 Completed Dag: dag_1463535189750_0001_1
2016-05-18 01:33:10,949 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535189750_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:10,950 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535189750_0001_1_post
2016-05-18 01:33:10,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:10,950 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:10,950 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:10,951 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, with id=dag_1463535189750_0001_1
2016-05-18 01:33:10,951 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3] pipeline/(1/1) ...tDeepPipeline/deeppipline, with id=dag_1463535189750_0001_1
2016-05-18 01:33:10,957 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...tDeepPipeline/deeppipline, applicationId=application_1463535189750_0001
2016-05-18 01:33:10,960 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:10,960 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:10,960 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:10,960 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:10,960 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:10,961 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:10,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:10,965 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:10,965 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:10,965 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:10,966 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:10,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:10,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:10,968 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:10,975 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/EB8AC9A9111C4033BDFD2DA4BDA98342/BEB785DF8CCA4696B537160A3C14F4B3/.tez/application_1463535189750_0001
2016-05-18 01:33:10,976 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline/_temporary
2016-05-18 01:33:10,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [pipeline] completed in: 00:01.246
2016-05-18 01:33:10,994 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:10,994 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:11,042 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testDeepPipeline/deeppipline
2016-05-18 01:33:11,080 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:11,081 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:11,116 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-18 01:33:11,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 01:33:11,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:11,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:11,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:11,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:33:11,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] parallel execution of steps is enabled: false
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] executing total steps: 1
2016-05-18 01:33:11,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] allocating management threads: 1
2016-05-18 01:33:11,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] starting step: (1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,150 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [split] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:11,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] tez session mode enabled: true
2016-05-18 01:33:11,167 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,168 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:11,168 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:11,176 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001 doesn't exist and is created
2016-05-18 01:33:11,198 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:11,198 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:11,199 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd
2016-05-18 01:33:11,214 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535191167_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,214 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,214 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:11,217 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:11,218 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:11,220 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:11,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:11,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:11,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:11,221 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:11,222 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:11,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:11,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:11,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,222 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,223 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:11,223 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:11,225 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:11,225 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:11,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:11,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir
2016-05-18 01:33:11,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/history.txt.appattempt_1463535191167_0001_000000, maxErrors=10
2016-05-18 01:33:11,226 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:11,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535191167_0001_000000, appSubmitTime=1463535191213, launchTime=1463535191214
2016-05-18 01:33:11,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:11,233 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40690
2016-05-18 01:33:11,236 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535191167
2016-05-18 01:33:11,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535191167_0001_000000, startTime=1463535191239
2016-05-18 01:33:11,239 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:11,299 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:11,299 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:11,299 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001, dagName=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,317 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,318 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:11,319 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535191167_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/dag_1463535191167_0001_1.dot
2016-05-18 01:33:11,319 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001_wd/localmode-log-dir/dag_1463535191167_0001_1-tez-dag.pb.txt
2016-05-18 01:33:11,323 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191167_0001_1
2016-05-18 01:33:11,323 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,323 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11 Running Dag: dag_1463535191167_0001_1
2016-05-18 01:33:11,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535191167_0001_1, submitTime=1463535191318
2016-05-18 01:33:11,338 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:11,338 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,338 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,339 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:11,339 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:11,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535191167_0001_1, initTime=1463535191339
2016-05-18 01:33:11,341 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:11,341 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:11,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_STARTED]: dagID=dag_1463535191167_0001_1, startTime=1463535191341
2016-05-18 01:33:11,341 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001, dagName=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate
2016-05-18 01:33:11,342 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:11,343 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D3350753BFB44335BB67E70CE86C1D0C : {A80D293996C5484BB2BD69789D8648CA={InputName=A80D293996C5484BB2BD69789D8648CA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:11,343 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: A80D293996C5484BB2BD69789D8648CA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:11,343 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,344 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,344 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:11,345 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: A80D293996C5484BB2BD69789D8648CA on vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,347 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input A80D293996C5484BB2BD69789D8648CA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:11,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:11,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:33:11,349 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:11,350 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:33:11,350 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: A80D293996C5484BB2BD69789D8648CA on vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,351 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:11,351 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {A80D293996C5484BB2BD69789D8648CA=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:11,352 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] parallelism set to 1
2016-05-18 01:33:11,352 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:11,353 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], numAdditionalOutputs=1
2016-05-18 01:33:11,353 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=963FB9B4FA5843BBB36D210BA6B2CFED, vertex=vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:11,355 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D3350753BFB44335BB67E70CE86C1D0C:963FB9B4FA5843BBB36D210BA6B2CFED using oldmapred API
2016-05-18 01:33:11,355 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=963FB9B4FA5843BBB36D210BA6B2CFED, vertexName=D3350753BFB44335BB67E70CE86C1D0C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:11,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, vertexId=vertex_1463535191167_0001_1_00, initRequestedTime=1463535191343, initedTime=1463535191353, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:11,356 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] with distanceFromRoot: 0
2016-05-18 01:33:11,356 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:11,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191167_0001_1_00, startRequestedTime=1463535191344, startedTime=1463535191357
2016-05-18 01:33:11,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191167_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:11,357 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:11,357 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D3350753BFB44335BB67E70CE86C1D0C
2016-05-18 01:33:11,358 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_STARTED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskId=task_1463535191167_0001_1_00_000000, scheduledTime=1463535191359, launchTime=1463535191359
2016-05-18 01:33:11,359 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,360 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:11,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191167_0001_00_000001, launchTime=1463535191362
2016-05-18 01:33:11,362 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,363 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] submitted tez dag to app master: application_1463535191167_0001, with dag id: dag_1463535191167_0001_1
2016-05-18 01:33:11,365 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191167_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11 Starting to run new task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,366 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:11,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, VertexName: D3350753BFB44335BB67E70CE86C1D0C, VertexParallelism: 1, TaskAttemptID:attempt_1463535191167_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A80D293996C5484BB2BD69789D8648CA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:11,367 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:11,367 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,368 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191167_0001_1_00_000000_0] started. Is using containerId: [container_1463535191167_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:11,368 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191167_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:11,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, startTime=1463535191368, containerId=container_1463535191167_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:11,373 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191167_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:11,373 INFO input.MRInput (MRInput.java:initialize(440)) - A80D293996C5484BB2BD69789D8648CA using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:11,374 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,379 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:11,385 INFO output.MROutput (MROutput.java:initialize(421)) - 963FB9B4FA5843BBB36D210BA6B2CFED: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:11,386 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, ordinal: 0
2016-05-18 01:33:11,386 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, mem on start (mb), free: 284, total: 394, max: 672
2016-05-18 01:33:11,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:11,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:11,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:11,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:11,387 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A80D293996C5484BB2BD69789D8648CA:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [963FB9B4FA5843BBB36D210BA6B2CFED:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:11,387 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:11,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:11,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:11,388 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A80D293996C5484BB2BD69789D8648CA:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:11,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:11,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A80D293996C5484BB2BD69789D8648CA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:11,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:11,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:11,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:11,389 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,393 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:33:11,393 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - A80D293996C5484BB2BD69789D8648CA initialized RecordReader from event
2016-05-18 01:33:11,393 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,394 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: A80D293996C5484BB2BD69789D8648CA
2016-05-18 01:33:11,394 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"], id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:11,394 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,394 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:33:11,395 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:11,398 INFO output.MROutput (MROutput.java:close(526)) - 963FB9B4FA5843BBB36D210BA6B2CFED closed
2016-05-18 01:33:11,398 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535191167_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:11,399 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,399 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3350753BFB44335BB67E70CE86C1D0C, mem on close (mb), free: 282, total: 394, max: 672
2016-05-18 01:33:11,400 INFO output.MROutput (MROutput.java:close(526)) - 963FB9B4FA5843BBB36D210BA6B2CFED closed
2016-05-18 01:33:11,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:11 Completed running task attempt: attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,400 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:11,400 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191167_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:11,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191167_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5][cascading.flow.SliceCounters Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=5]]
2016-05-18 01:33:11,402 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191167_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:11,402 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:11,402 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:11,402 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskAttemptId=attempt_1463535191167_0001_1_00_000000_0, creationTime=1463535191359, allocationTime=1463535191361, startTime=1463535191368, finishTime=1463535191402, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:11,403 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191167_0001_1_00_000000_0
2016-05-18 01:33:11,407 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:TASK_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, taskId=task_1463535191167_0001_1_00_000000, startTime=1463535191368, finishTime=1463535191403, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463535191167_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:11,407 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:11,407 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C]
2016-05-18 01:33:11,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:VERTEX_FINISHED]: vertexName=D3350753BFB44335BB67E70CE86C1D0C, vertexId=vertex_1463535191167_0001_1_00, initRequestedTime=1463535191343, initedTime=1463535191353, startRequestedTime=1463535191344, startedTime=1463535191357, finishTime=1463535191407, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535191368, firstTasksToStart=[ task_1463535191167_0001_1_00_000000 ], lastTaskFinishTime=1463535191402, lastTasksToFinish=[ task_1463535191167_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191167_0001_1_00_000000 ], longestDurationTasks=[ task_1463535191167_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:11,409 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:11,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191167_0001_00_000001, stoppedTime=1463535191409, exitStatus=0
2016-05-18 01:33:11,409 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:11,409 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:11,410 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535191167_0001_1
2016-05-18 01:33:11,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535191167_0001_1
2016-05-18 01:33:11,410 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191167_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:11,411 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:11,410 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 963FB9B4FA5843BBB36D210BA6B2CFED for vertex: vertex_1463535191167_0001_1_00 [D3350753BFB44335BB67E70CE86C1D0C], outputName: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:11,411 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191167_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:11,411 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191167_0001_00_000001, Exiting
2016-05-18 01:33:11,411 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,412 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191167_0001_00_000001
2016-05-18 01:33:11,414 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191167_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:11,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191167_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:11,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191167_0001_00_000001 completed successfully
2016-05-18 01:33:11,422 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:963FB9B4FA5843BBB36D210BA6B2CFED of vertex/vertexGroup:D3350753BFB44335BB67E70CE86C1D0C isVertexGroupOutput:false
2016-05-18 01:33:11,422 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:11,423 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535191167_0001_1, queueSize=0
2016-05-18 01:33:11,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191167_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535191167_0001_1, startTime=1463535191341, finishTime=1463535191422, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, cascading.flow.SliceCounters, Process_Begin_Time=1463535191395, Process_Duration=4, Process_End_Time=1463535191399, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:11,424 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535191167_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:11,424 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191167_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:11 Completed Dag: dag_1463535191167_0001_1
2016-05-18 01:33:11,425 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535191167_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:11,425 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191167_0001_1_post
2016-05-18 01:33:11,425 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:11,425 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:11,425 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:11,425 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, with id=dag_1463535191167_0001_1
2016-05-18 01:33:11,425 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767] split/(1/1) ...litPipe/splitintermediate, with id=dag_1463535191167_0001_1
2016-05-18 01:33:11,440 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...litPipe/splitintermediate, applicationId=application_1463535191167_0001
2016-05-18 01:33:11,442 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:11,442 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:11,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:11,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:11,442 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:11,443 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:11,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:11,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:11,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:11,447 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:11,447 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:11,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:11,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:11,450 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:11,455 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/3A5F5169761A42F6B6F35EE5DC13C69E/AD3A0230319F4FF780983D98621E6767/.tez/application_1463535191167_0001
2016-05-18 01:33:11,456 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/_temporary
2016-05-18 01:33:11,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [split] completed in: 00:00.307
2016-05-18 01:33:11,471 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:33:11,471 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:33:11,486 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:11,487 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:11,535 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 01:33:11,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 01:33:11,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:11,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:11,544 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:11,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:11,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit
2016-05-18 01:33:11,624 INFO tez.Hadoop2TezFlowStep (Hadoop2TezFlowStep.java:createDAG(265)) - adding grouped edge between: A24AE922279248E69EE6985AE886DB2D,FE62E2013A1A4DE5B2359A2A3196EDC1 and F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:11,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] starting
2016-05-18 01:33:11,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:11,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] sink: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"]
2016-05-18 01:33:11,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] parallel execution of steps is enabled: false
2016-05-18 01:33:11,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] executing total steps: 1
2016-05-18 01:33:11,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] allocating management threads: 1
2016-05-18 01:33:11,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] starting step: (1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:11,629 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lhs+rhs] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:11,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] tez session mode enabled: true
2016-05-18 01:33:11,651 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,652 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:11,652 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:11,662 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001 doesn't exist and is created
2016-05-18 01:33:11,683 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:11,683 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:11,684 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd
2016-05-18 01:33:11,701 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535191652_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:11,702 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,702 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:11,704 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:11,704 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:11,705 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:11,707 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:11,707 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:11,708 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:11,708 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:11,708 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:11,708 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:11,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:11,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:11,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:11,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:11,710 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:11,712 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:11,713 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:11,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:11,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir
2016-05-18 01:33:11,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/history.txt.appattempt_1463535191652_0001_000000, maxErrors=10
2016-05-18 01:33:11,715 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:11,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535191652_0001_000000, appSubmitTime=1463535191701, launchTime=1463535191701
2016-05-18 01:33:11,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:11,730 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35647
2016-05-18 01:33:11,730 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535191652
2016-05-18 01:33:11,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535191652_0001_000000, startTime=1463535191737
2016-05-18 01:33:11,738 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:11,783 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:11,784 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:11,784 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001, dagName=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,012 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,013 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:12,013 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535191652_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/dag_1463535191652_0001_1.dot
2016-05-18 01:33:12,014 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-log-dir/dag_1463535191652_0001_1-tez-dag.pb.txt
2016-05-18 01:33:12,029 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191652_0001_1
2016-05-18 01:33:12,029 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,029 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12 Running Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535191652_0001_1, submitTime=1463535192012
2016-05-18 01:33:12,047 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,047 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,047 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,048 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,048 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,048 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,051 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001/tez-conf.pb" } size: 40938 timestamp: 1463535191000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,051 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,051 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,051 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:12,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535191652_0001_1, initTime=1463535192053
2016-05-18 01:33:12,053 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:12,054 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:12,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_STARTED]: dagID=dag_1463535191652_0001_1, startTime=1463535192054
2016-05-18 01:33:12,054 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:12,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FE62E2013A1A4DE5B2359A2A3196EDC1 : {963FB9B4FA5843BBB36D210BA6B2CFED={InputName=963FB9B4FA5843BBB36D210BA6B2CFED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 963FB9B4FA5843BBB36D210BA6B2CFED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,054 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,055 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,055 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A24AE922279248E69EE6985AE886DB2D : {963FB9B4FA5843BBB36D210BA6B2CFED={InputName=963FB9B4FA5843BBB36D210BA6B2CFED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,056 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 963FB9B4FA5843BBB36D210BA6B2CFED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,056 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,056 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,057 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,057 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001, dagName=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit
2016-05-18 01:33:12,058 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,058 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,061 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 963FB9B4FA5843BBB36D210BA6B2CFED asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,062 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,062 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,062 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,064 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], numAdditionalOutputs=1
2016-05-18 01:33:12,064 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F1CC6B9FD79F49E0BAE2E82A62FDA501, vertex=vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:12,065 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 963FB9B4FA5843BBB36D210BA6B2CFED asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,068 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F7C5BC8A38EB422FAAA1BF2B645CE780:F1CC6B9FD79F49E0BAE2E82A62FDA501 using oldmapred API
2016-05-18 01:33:12,068 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F1CC6B9FD79F49E0BAE2E82A62FDA501, vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:12,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, vertexId=vertex_1463535191652_0001_1_02, initRequestedTime=1463535192057, initedTime=1463535192064, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,074 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] submitted tez dag to app master: application_1463535191652_0001, with dag id: dag_1463535191652_0001_1
2016-05-18 01:33:12,085 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,085 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:33:12,085 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,085 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,086 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,087 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,087 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {963FB9B4FA5843BBB36D210BA6B2CFED=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,088 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] parallelism set to 1
2016-05-18 01:33:12,088 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A24AE922279248E69EE6985AE886DB2D, vertexId=vertex_1463535191652_0001_1_01, initRequestedTime=1463535192055, initedTime=1463535192089, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,089 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] with distanceFromRoot: 0
2016-05-18 01:33:12,089 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_01, startRequestedTime=1463535192057, startedTime=1463535192089
2016-05-18 01:33:12,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,089 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,090 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535191652_0001_1_01 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 1 numSources: 2
2016-05-18 01:33:12,090 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 1 numSources: 2
2016-05-18 01:33:12,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 20 Min splitLength: 52428800 New desired splits: 1 Total length: 20 Original splits: 2
2016-05-18 01:33:12,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 20 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 20 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,092 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 963FB9B4FA5843BBB36D210BA6B2CFED on vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,092 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,092 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {963FB9B4FA5843BBB36D210BA6B2CFED=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,093 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] parallelism set to 1
2016-05-18 01:33:12,093 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, vertexId=vertex_1463535191652_0001_1_00, initRequestedTime=1463535192054, initedTime=1463535192093, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,093 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] with distanceFromRoot: 0
2016-05-18 01:33:12,094 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_00, startRequestedTime=1463535192057, startedTime=1463535192094
2016-05-18 01:33:12,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535191652_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,094 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,094 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535191652_0001_1_00 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] numStartedSources: 2 numSources: 2
2016-05-18 01:33:12,094 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] with distanceFromRoot: 1
2016-05-18 01:33:12,095 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,095 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,095 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,095 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535191652_0001_1_02, startRequestedTime=1463535192094, startedTime=1463535192094
2016-05-18 01:33:12,097 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskId=task_1463535191652_0001_1_00_000000, scheduledTime=1463535192098, launchTime=1463535192098
2016-05-18 01:33:12,098 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskId=task_1463535191652_0001_1_01_000000, scheduledTime=1463535192098, launchTime=1463535192098
2016-05-18 01:33:12,099 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,103 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:12,103 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,104 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FE62E2013A1A4DE5B2359A2A3196EDC1 in vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,105 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A24AE922279248E69EE6985AE886DB2D in vertex: F7C5BC8A38EB422FAAA1BF2B645CE780 numBipartiteSourceTasks: 2
2016-05-18 01:33:12,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000001, launchTime=1463535192106
2016-05-18 01:33:12,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000002, launchTime=1463535192106
2016-05-18 01:33:12,107 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,110 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,111 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_00_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, startTime=1463535192110, containerId=container_1463535191652_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:12,111 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,112 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_01_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,112 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, startTime=1463535192112, containerId=container_1463535191652_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,113 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,113 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,113 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: FE62E2013A1A4DE5B2359A2A3196EDC1, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,114 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,114 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: A24AE922279248E69EE6985AE886DB2D, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=963FB9B4FA5843BBB36D210BA6B2CFED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,116 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,118 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191652_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:12,118 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,122 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,124 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,124 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535191652_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:33:12,125 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,132 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,136 INFO input.MRInput (MRInput.java:initialize(440)) - 963FB9B4FA5843BBB36D210BA6B2CFED using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,136 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,136 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,138 INFO input.MRInput (MRInput.java:initialize(440)) - 963FB9B4FA5843BBB36D210BA6B2CFED using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,139 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,141 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, ordinal: 1
2016-05-18 01:33:12,141 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, mem on start (mb), free: 171, total: 394, max: 672
2016-05-18 01:33:12,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,142 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[963FB9B4FA5843BBB36D210BA6B2CFED:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F7C5BC8A38EB422FAAA1BF2B645CE780:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,142 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,143 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F7C5BC8A38EB422FAAA1BF2B645CE780:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 963FB9B4FA5843BBB36D210BA6B2CFED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,147 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,147 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A24AE922279248E69EE6985AE886DB2D, ordinal: 0
2016-05-18 01:33:12,147 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24AE922279248E69EE6985AE886DB2D, mem on start (mb), free: 171, total: 394, max: 672
2016-05-18 01:33:12,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,148 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[963FB9B4FA5843BBB36D210BA6B2CFED:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F7C5BC8A38EB422FAAA1BF2B645CE780:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,148 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,149 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[963FB9B4FA5843BBB36D210BA6B2CFED:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F7C5BC8A38EB422FAAA1BF2B645CE780:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 963FB9B4FA5843BBB36D210BA6B2CFED being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,151 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,151 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"] streamed: true, id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,152 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']], id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,152 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,154 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,155 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F7C5BC8A38EB422FAAA1BF2B645CE780 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@38c108c8, comparator=cascading.tuple.hadoop.util.TupleComparator@9279f94, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,155 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false
2016-05-18 01:33:12,171 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:33:12,171 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 963FB9B4FA5843BBB36D210BA6B2CFED initialized RecordReader from event
2016-05-18 01:33:12,182 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,182 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,183 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,183 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F7C5BC8A38EB422FAAA1BF2B645CE780: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12,183 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,183 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,186 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,187 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"] streamed: true, id: 963FB9B4FA5843BBB36D210BA6B2CFED
2016-05-18 01:33:12,187 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']], id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,187 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,189 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:0+10, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate/part-v000-o000-00000:10+10], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=20}
2016-05-18 01:33:12,189 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 963FB9B4FA5843BBB36D210BA6B2CFED initialized RecordReader from event
2016-05-18 01:33:12,190 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F7C5BC8A38EB422FAAA1BF2B645CE780 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@13432534, comparator=cascading.tuple.hadoop.util.TupleComparator@1d3f82cd, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,191 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false
2016-05-18 01:33:12,192 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,192 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE62E2013A1A4DE5B2359A2A3196EDC1, mem on close (mb), free: 144, total: 394, max: 672
2016-05-18 01:33:12,192 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Starting flush of map output
2016-05-18 01:33:12,192 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Span0.length = 5, perItem = 15
2016-05-18 01:33:12,192 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F7C5BC8A38EB422FAAA1BF2B645CE780: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,192 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:12,193 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_00_000000_0_10072_0/file.out
2016-05-18 01:33:12,207 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,208 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,208 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F7C5BC8A38EB422FAAA1BF2B645CE780: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,207 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12,208 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F7C5BC8A38EB422FAAA1BF2B645CE780: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,210 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate"]
2016-05-18 01:33:12,212 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:12,214 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:12,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,213 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A24AE922279248E69EE6985AE886DB2D, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,218 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,218 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskAttemptId=attempt_1463535191652_0001_1_00_000000_0, creationTime=1463535192098, allocationTime=1463535192104, startTime=1463535192110, finishTime=1463535192218, timeTaken=108, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,219 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_00_000000_0
2016-05-18 01:33:12,218 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:12,219 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, taskId=task_1463535191652_0001_1_00_000000, startTime=1463535192110, finishTime=1463535192219, timeTaken=109, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,220 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,220 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,220 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, vertexId=vertex_1463535191652_0001_1_00, initRequestedTime=1463535192054, initedTime=1463535192093, startRequestedTime=1463535192057, startedTime=1463535192094, finishTime=1463535192220, timeTaken=126, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192184, Process_Duration=8, Process_End_Time=1463535192192, Read_Duration=6, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192110, firstTasksToStart=[ task_1463535191652_0001_1_00_000000 ], lastTaskFinishTime=1463535192218, lastTasksToFinish=[ task_1463535191652_0001_1_00_000000 ], minTaskDuration=108, maxTaskDuration=108, avgTaskDuration=108.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_00_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,222 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,222 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] attempt: attempt_1463535191652_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000001, stoppedTime=1463535192223, exitStatus=0
2016-05-18 01:33:12,224 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,224 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,225 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,224 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,225 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A24AE922279248E69EE6985AE886DB2D, mem on close (mb), free: 141, total: 394, max: 672
2016-05-18 01:33:12,225 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Starting flush of map output
2016-05-18 01:33:12,226 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Span0.length = 5, perItem = 15
2016-05-18 01:33:12,226 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F7C5BC8A38EB422FAAA1BF2B645CE780: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,227 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:33:12,228 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F7C5BC8A38EB422FAAA1BF2B645CE780: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_01_000000_0_10074_0/file.out
2016-05-18 01:33:12,231 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,231 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,231 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000001, Exiting
2016-05-18 01:33:12,231 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,231 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000001
2016-05-18 01:33:12,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12,236 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,236 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000001 completed successfully
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,237 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:12,237 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:12,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,239 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,239 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,239 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:12,239 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskAttemptId=attempt_1463535191652_0001_1_01_000000_0, creationTime=1463535192099, allocationTime=1463535192104, startTime=1463535192112, finishTime=1463535192240, timeTaken=128, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,240 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,240 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_01_000000_0
2016-05-18 01:33:12,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, taskId=task_1463535191652_0001_1_01_000000, startTime=1463535192112, finishTime=1463535192240, timeTaken=128, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,241 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,241 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=A24AE922279248E69EE6985AE886DB2D, vertexId=vertex_1463535191652_0001_1_01, initRequestedTime=1463535192055, initedTime=1463535192089, startRequestedTime=1463535192057, startedTime=1463535192089, finishTime=1463535192241, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192216, Process_Duration=7, Process_End_Time=1463535192223, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192112, firstTasksToStart=[ task_1463535191652_0001_1_01_000000 ], lastTaskFinishTime=1463535192240, lastTasksToFinish=[ task_1463535191652_0001_1_01_000000 ], minTaskDuration=128, maxTaskDuration=128, avgTaskDuration=128.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_01_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,242 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000002, stoppedTime=1463535192243, exitStatus=0
2016-05-18 01:33:12,243 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] attempt: attempt_1463535191652_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,243 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,243 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,243 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:12,243 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,244 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A24AE922279248E69EE6985AE886DB2D, destinationVertexName=F7C5BC8A38EB422FAAA1BF2B645CE780. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,244 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,244 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_STARTED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskId=task_1463535191652_0001_1_02_000000, scheduledTime=1463535192246, launchTime=1463535192246
2016-05-18 01:33:12,247 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,248 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,248 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,249 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000002, Exiting
2016-05-18 01:33:12,249 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,249 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000002
2016-05-18 01:33:12,250 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,250 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000002 completed successfully
2016-05-18 01:33:12,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535191652_0001_00_000003, launchTime=1463535192250
2016-05-18 01:33:12,252 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,252 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535191652_0001_1_02_000000_0] started. Is using containerId: [container_1463535191652_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, startTime=1463535192252, containerId=container_1463535191652_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,253 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,253 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, VertexName: F7C5BC8A38EB422FAAA1BF2B645CE780, VertexParallelism: 1, TaskAttemptID:attempt_1463535191652_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE62E2013A1A4DE5B2359A2A3196EDC1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=A24AE922279248E69EE6985AE886DB2D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F1CC6B9FD79F49E0BAE2E82A62FDA501, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] groupInputSpecList=[{Group: F7C5BC8A38EB422FAAA1BF2B645CE780 { A24AE922279248E69EE6985AE886DB2D FE62E2013A1A4DE5B2359A2A3196EDC1 } MergedInputDescriptor: org.apache.tez.runtime.library.input.OrderedGroupedMergedKVInput}, ]
2016-05-18 01:33:12,254 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,256 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,257 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535191652_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,258 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535191652_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:33:12,262 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,267 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,274 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, ordinal: 2
2016-05-18 01:33:12,276 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, mem on start (mb), free: 134, total: 394, max: 672
2016-05-18 01:33:12,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:33:12,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,275 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,281 INFO output.MROutput (MROutput.java:initialize(421)) - F1CC6B9FD79F49E0BAE2E82A62FDA501: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:12,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,284 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FE62E2013A1A4DE5B2359A2A3196EDC1:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [F1CC6B9FD79F49E0BAE2E82A62FDA501:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [A24AE922279248E69EE6985AE886DB2D:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:12,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:33:12,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:33:12,286 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FE62E2013A1A4DE5B2359A2A3196EDC1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [F1CC6B9FD79F49E0BAE2E82A62FDA501:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [A24AE922279248E69EE6985AE886DB2D:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:12,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(278)) - Ignoring FE62E2013A1A4DE5B2359A2A3196EDC1 for start, since it will be controlled via it's Group
2016-05-18 01:33:12,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(278)) - Ignoring A24AE922279248E69EE6985AE886DB2D for start, since it will be controlled via it's Group
2016-05-18 01:33:12,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(298)) - InputGroup: F7C5BC8A38EB422FAAA1BF2B645CE780 being auto started by the framework. Subsequent instance will not be auto-started
2016-05-18 01:33:12,287 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A24AE922279248E69EE6985AE886DB2D: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,288 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A24AE922279248E69EE6985AE886DB2D: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,290 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A24AE922279248E69EE6985AE886DB2D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,293 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,293 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FE62E2013A1A4DE5B2359A2A3196EDC1: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,295 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FE62E2013A1A4DE5B2359A2A3196EDC1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F7C5BC8A38EB422FAAA1BF2B645CE780
2016-05-18 01:33:12,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,300 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,301 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FE62E2013A1A4DE5B2359A2A3196EDC1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,303 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A24AE922279248E69EE6985AE886DB2D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,315 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2016-05-18 01:33:12,315 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] streamed: true, id: 1FC472054B794F68A07829557F3C354B
2016-05-18 01:33:12,315 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"], id: F1CC6B9FD79F49E0BAE2E82A62FDA501
2016-05-18 01:33:12,316 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535191652_0001_1_00_000000_0_10072} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535192315, TimeTaken=1, Rate=0.09 MB/s
2016-05-18 01:33:12,316 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,316 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit"]
2016-05-18 01:33:12,316 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedMergedKVInput#start() on: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']] 1FC472054B794F68A07829557F3C354B, for 1 inputs
2016-05-18 01:33:12,316 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,317 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FE62E2013A1A4DE5B2359A2A3196EDC1, wasInterrupted=false
2016-05-18 01:33:12,317 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,318 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:12,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_00_000000_0_10072/file.out, len=95, isLocal=true
2016-05-18 01:33:12,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:12,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:12,319 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:12,319 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:12,319 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,320 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Shuffle Runner thread complete
2016-05-18 01:33:12,320 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - FE62E2013A1A4DE5B2359A2A3196EDC1: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:12,320 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535191652_0001_1_01_000000_0_10074} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463535192320, TimeTaken=17, Rate=0.01 MB/s
2016-05-18 01:33:12,321 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,321 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,321 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A24AE922279248E69EE6985AE886DB2D, wasInterrupted=false
2016-05-18 01:33:12,322 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:12,323 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:12,323 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001_wd/localmode-local-dir/output/attempt_1463535191652_0001_1_01_000000_0_10074/file.out, len=95, isLocal=true
2016-05-18 01:33:12,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:33:12,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:12,324 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:12,324 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:33:12,324 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,325 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A24AE922279248E69EE6985AE886DB2D: Shuffle Runner thread complete
2016-05-18 01:33:12,325 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, all 1 inputs ready in: 00:00:00.008
2016-05-18 01:33:12,326 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A24AE922279248E69EE6985AE886DB2D: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:12,328 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FE62E2013A1A4DE5B2359A2A3196EDC1: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:12,332 INFO output.MROutput (MROutput.java:close(526)) - F1CC6B9FD79F49E0BAE2E82A62FDA501 closed
2016-05-18 01:33:12,333 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535191652_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:12,334 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,334 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7C5BC8A38EB422FAAA1BF2B645CE780, mem on close (mb), free: 342, total: 390, max: 672
2016-05-18 01:33:12,334 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FE62E2013A1A4DE5B2359A2A3196EDC1
2016-05-18 01:33:12,334 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FE62E2013A1A4DE5B2359A2A3196EDC1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:12,334 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A24AE922279248E69EE6985AE886DB2D
2016-05-18 01:33:12,334 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A24AE922279248E69EE6985AE886DB2D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:12,334 INFO output.MROutput (MROutput.java:close(526)) - F1CC6B9FD79F49E0BAE2E82A62FDA501 closed
2016-05-18 01:33:12,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,335 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:12,335 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535191652_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:12,336 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535191652_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=10]]
2016-05-18 01:33:12,338 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535191652_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskAttemptId=attempt_1463535191652_0001_1_02_000000_0, creationTime=1463535192247, allocationTime=1463535192248, startTime=1463535192252, finishTime=1463535192338, timeTaken=86, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,338 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,338 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:12,338 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,338 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535191652_0001_1_02_000000_0
2016-05-18 01:33:12,339 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:TASK_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, taskId=task_1463535191652_0001_1_02_000000, startTime=1463535192252, finishTime=1463535192338, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1463535191652_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10
2016-05-18 01:33:12,339 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,339 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780]
2016-05-18 01:33:12,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:VERTEX_FINISHED]: vertexName=F7C5BC8A38EB422FAAA1BF2B645CE780, vertexId=vertex_1463535191652_0001_1_02, initRequestedTime=1463535192057, initedTime=1463535192064, startRequestedTime=1463535192094, startedTime=1463535192094, finishTime=1463535192340, timeTaken=246, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=10, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535192325, Process_Duration=9, Process_End_Time=1463535192334, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535192252, firstTasksToStart=[ task_1463535191652_0001_1_02_000000 ], lastTaskFinishTime=1463535192338, lastTasksToFinish=[ task_1463535191652_0001_1_02_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535191652_0001_1_02_000000 ], longestDurationTasks=[ task_1463535191652_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,341 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535191652_0001_00_000003, stoppedTime=1463535192341, exitStatus=0
2016-05-18 01:33:12,341 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,341 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,341 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535191652_0001_1_00 [FE62E2013A1A4DE5B2359A2A3196EDC1]
2016-05-18 01:33:12,342 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535191652_0001_1_01 [A24AE922279248E69EE6985AE886DB2D]
2016-05-18 01:33:12,342 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535191652_0001_1
2016-05-18 01:33:12,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535191652_0001_1
2016-05-18 01:33:12,342 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F1CC6B9FD79F49E0BAE2E82A62FDA501 for vertex: vertex_1463535191652_0001_1_02 [F7C5BC8A38EB422FAAA1BF2B645CE780], outputName: F1CC6B9FD79F49E0BAE2E82A62FDA501
2016-05-18 01:33:12,345 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:12,345 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535191652_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,349 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535191652_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,349 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535191652_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,350 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535191652_0001_00_000003, Exiting
2016-05-18 01:33:12,350 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,350 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535191652_0001_00_000003
2016-05-18 01:33:12,350 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535191652_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12,350 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535191652_0001_00_000003 completed successfully
2016-05-18 01:33:12,353 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F1CC6B9FD79F49E0BAE2E82A62FDA501 of vertex/vertexGroup:F7C5BC8A38EB422FAAA1BF2B645CE780 isVertexGroupOutput:false
2016-05-18 01:33:12,353 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,356 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535191652_0001_1, queueSize=0
2016-05-18 01:33:12,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535191652_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535191652_0001_1, startTime=1463535192054, finishTime=1463535192353, timeTaken=299, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=780, AM_GC_TIME_MILLIS=25, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=57, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605576725, Process_Duration=24, Process_End_Time=4390605576749, Read_Duration=13, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:33:12,356 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535191652_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:12,356 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535191652_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:12 Completed Dag: dag_1463535191652_0001_1
2016-05-18 01:33:12,357 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535191652_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:12,357 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535191652_0001_1_post
2016-05-18 01:33:12,358 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,358 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:12,358 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:12,358 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, with id=dag_1463535191652_0001_1
2016-05-18 01:33:12,358 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560] lhs+rhs/(1/1) ...pBySplitPipe/groupbysplit, with id=dag_1463535191652_0001_1
2016-05-18 01:33:12,363 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pBySplitPipe/groupbysplit, applicationId=application_1463535191652_0001
2016-05-18 01:33:12,365 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:12,365 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:12,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:12,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:12,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:12,366 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:12,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:12,367 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:12,367 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:12,367 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:12,368 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:12,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:12,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:12,368 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:12,372 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/71F6633A48944E558585375333748DA9/AE97B355DEAA4348991E6F26D7E15560/.tez/application_1463535191652_0001
2016-05-18 01:33:12,373 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit/_temporary
2016-05-18 01:33:12,373 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lhs+rhs] completed in: 00:00.745
2016-05-18 01:33:12,448 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/splitintermediate
2016-05-18 01:33:12,448 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testGroupBySplitPipe/groupbysplit
2016-05-18 01:33:12,473 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:12,473 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:12,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:33:12,523 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 01:33:12,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:12,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:33:12,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:12,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:12,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars
2016-05-18 01:33:12,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] starting
2016-05-18 01:33:12,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"]
2016-05-18 01:33:12,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] parallel execution of steps is enabled: false
2016-05-18 01:33:12,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] executing total steps: 1
2016-05-18 01:33:12,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] allocating management threads: 1
2016-05-18 01:33:12,669 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] starting step: (1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,669 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [**&%&%bar:bar@foo://bl...] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:12,683 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] tez session mode enabled: true
2016-05-18 01:33:12,689 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:12,689 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:12,689 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:12,700 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001 doesn't exist and is created
2016-05-18 01:33:12,723 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:12,723 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:12,724 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd
2016-05-18 01:33:12,742 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535192689_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:12,742 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:12,743 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:12,744 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:12,744 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:12,745 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:12,746 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:12,746 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:12,747 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:12,747 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:12,747 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:12,747 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:12,747 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:12,748 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:12,749 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:12,750 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:12,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:12,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir
2016-05-18 01:33:12,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/history.txt.appattempt_1463535192689_0001_000000, maxErrors=10
2016-05-18 01:33:12,751 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:12,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535192689_0001_000000, appSubmitTime=1463535192742, launchTime=1463535192742
2016-05-18 01:33:12,754 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:12,761 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50150
2016-05-18 01:33:12,761 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535192689
2016-05-18 01:33:12,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535192689_0001_000000, startTime=1463535192763
2016-05-18 01:33:12,763 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:12,823 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:12,823 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:12,823 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001, dagName=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,841 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,841 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:12,842 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535192689_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/dag_1463535192689_0001_1.dot
2016-05-18 01:33:12,843 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-log-dir/dag_1463535192689_0001_1-tez-dag.pb.txt
2016-05-18 01:33:12,853 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535192689_0001_1
2016-05-18 01:33:12,853 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,854 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12 Running Dag: dag_1463535192689_0001_1
2016-05-18 01:33:12,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535192689_0001_1, submitTime=1463535192841
2016-05-18 01:33:12,869 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,869 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,869 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,870 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,870 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,872 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001/tez-conf.pb" } size: 40938 timestamp: 1463535192000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:12,872 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,872 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:12,873 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:12,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535192689_0001_1, initTime=1463535192873
2016-05-18 01:33:12,874 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:12,874 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:12,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_STARTED]: dagID=dag_1463535192689_0001_1, startTime=1463535192874
2016-05-18 01:33:12,874 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:12,874 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B243B3E306934EDA844CBCFB2DC403CD : {35601625C6C44CF4B4693503323B1B33={InputName=35601625C6C44CF4B4693503323B1B33}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 35601625C6C44CF4B4693503323B1B33, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,875 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:12,876 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,876 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 35601625C6C44CF4B4693503323B1B33 on vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,879 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 35601625C6C44CF4B4693503323B1B33 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:12,879 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,879 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,881 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,881 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001, dagName=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars
2016-05-18 01:33:12,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, vertexId=vertex_1463535192689_0001_1_01, initRequestedTime=1463535192876, initedTime=1463535192881, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,882 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,882 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,882 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:12,882 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:12,882 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:12,883 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:12,883 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 35601625C6C44CF4B4693503323B1B33 on vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,887 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:33:12,887 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,887 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:12,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:33:12,888 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], numAdditionalOutputs=1
2016-05-18 01:33:12,888 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=29A042B03F7B4091B3E9D134E2EE9553, vertex=vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:12,890 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A5DFDE462F5E409084AA77724F0033C8:29A042B03F7B4091B3E9D134E2EE9553 using oldmapred API
2016-05-18 01:33:12,891 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=29A042B03F7B4091B3E9D134E2EE9553, vertexName=A5DFDE462F5E409084AA77724F0033C8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:12,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, vertexId=vertex_1463535192689_0001_1_02, initRequestedTime=1463535192882, initedTime=1463535192888, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:33:12,892 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:33:12,893 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,893 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {35601625C6C44CF4B4693503323B1B33=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:12,894 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] parallelism set to 1
2016-05-18 01:33:12,894 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:12,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, vertexId=vertex_1463535192689_0001_1_00, initRequestedTime=1463535192874, initedTime=1463535192894, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:12,894 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] with distanceFromRoot: 0
2016-05-18 01:33:12,895 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:12,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_00, startRequestedTime=1463535192875, startedTime=1463535192895
2016-05-18 01:33:12,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535192689_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:12,895 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,895 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535192689_0001_1_00 for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] numStartedSources: 1 numSources: 1
2016-05-18 01:33:12,895 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] with distanceFromRoot: 1
2016-05-18 01:33:12,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_01, startRequestedTime=1463535192895, startedTime=1463535192895
2016-05-18 01:33:12,896 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,896 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463535192689_0001_1_01 for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] numStartedSources: 1 numSources: 1
2016-05-18 01:33:12,896 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] with distanceFromRoot: 2
2016-05-18 01:33:12,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535192689_0001_1_02, startRequestedTime=1463535192896, startedTime=1463535192897
2016-05-18 01:33:12,897 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:12,901 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: FDC9A2FF27E24C74964FCCD237F4FB29 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,901 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:12,902 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,901 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B243B3E306934EDA844CBCFB2DC403CD in vertex: FDC9A2FF27E24C74964FCCD237F4FB29 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskId=task_1463535192689_0001_1_00_000000, scheduledTime=1463535192902, launchTime=1463535192902
2016-05-18 01:33:12,903 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,903 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A5DFDE462F5E409084AA77724F0033C8 with 0 source tasks and 1 pending tasks
2016-05-18 01:33:12,903 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:12,904 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FDC9A2FF27E24C74964FCCD237F4FB29 in vertex: A5DFDE462F5E409084AA77724F0033C8 numBipartiteSourceTasks: 1
2016-05-18 01:33:12,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000001, launchTime=1463535192906
2016-05-18 01:33:12,906 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,907 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_00_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, startTime=1463535192907, containerId=container_1463535192689_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:12,908 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,909 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] submitted tez dag to app master: application_1463535192689_0001, with dag id: dag_1463535192689_0001_1
2016-05-18 01:33:12,909 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,909 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,912 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: B243B3E306934EDA844CBCFB2DC403CD, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=35601625C6C44CF4B4693503323B1B33, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,912 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,912 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,913 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535192689_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:12,913 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:12,915 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,916 INFO input.MRInput (MRInput.java:initialize(440)) - 35601625C6C44CF4B4693503323B1B33 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:12,917 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,919 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, ordinal: 0
2016-05-18 01:33:12,919 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, mem on start (mb), free: 209, total: 390, max: 672
2016-05-18 01:33:12,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,922 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[35601625C6C44CF4B4693503323B1B33:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FDC9A2FF27E24C74964FCCD237F4FB29:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,922 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:33:12,923 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[35601625C6C44CF4B4693503323B1B33:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FDC9A2FF27E24C74964FCCD237F4FB29:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 35601625C6C44CF4B4693503323B1B33 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,925 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,927 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:12,927 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 35601625C6C44CF4B4693503323B1B33 initialized RecordReader from event
2016-05-18 01:33:12,929 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,929 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 35601625C6C44CF4B4693503323B1B33
2016-05-18 01:33:12,929 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]], id: A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:12,929 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:12,931 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - FDC9A2FF27E24C74964FCCD237F4FB29 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@66d1439c, comparator=cascading.tuple.hadoop.util.TupleComparator@34adeb80, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:12,931 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for FDC9A2FF27E24C74964FCCD237F4FB29: , UsingHashComparator=false
2016-05-18 01:33:12,955 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:12,956 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:12,956 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for FDC9A2FF27E24C74964FCCD237F4FB29: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:12,956 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - FDC9A2FF27E24C74964FCCD237F4FB29: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:12,956 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:12,957 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:12,959 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,959 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B243B3E306934EDA844CBCFB2DC403CD, mem on close (mb), free: 244, total: 393, max: 672
2016-05-18 01:33:12,959 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - FDC9A2FF27E24C74964FCCD237F4FB29: Starting flush of map output
2016-05-18 01:33:12,959 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - FDC9A2FF27E24C74964FCCD237F4FB29: Span0.length = 5, perItem = 13
2016-05-18 01:33:12,959 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - FDC9A2FF27E24C74964FCCD237F4FB29: done sorting span=0, length=5, time=0
2016-05-18 01:33:12,961 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - FDC9A2FF27E24C74964FCCD237F4FB29: Heap = SpanIterator<0:4> (span=Span[16777216,65]),
2016-05-18 01:33:12,962 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - FDC9A2FF27E24C74964FCCD237F4FB29: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_00_000000_0_10081_0/file.out
2016-05-18 01:33:12,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:12 Completed running task attempt: attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,965 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:12,965 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:12,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:33:12,967 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskAttemptId=attempt_1463535192689_0001_1_00_000000_0, creationTime=1463535192902, allocationTime=1463535192905, startTime=1463535192907, finishTime=1463535192967, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:12,967 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_00_000000_0
2016-05-18 01:33:12,967 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:12,967 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:12,968 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, taskId=task_1463535192689_0001_1_00_000000, startTime=1463535192907, finishTime=1463535192967, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:33:12,968 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,968 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:12,968 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:12,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=B243B3E306934EDA844CBCFB2DC403CD, vertexId=vertex_1463535192689_0001_1_00, initRequestedTime=1463535192874, initedTime=1463535192894, startRequestedTime=1463535192875, startedTime=1463535192895, finishTime=1463535192968, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463535192957, Process_Duration=2, Process_End_Time=1463535192959, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463535192907, firstTasksToStart=[ task_1463535192689_0001_1_00_000000 ], lastTaskFinishTime=1463535192967, lastTasksToFinish=[ task_1463535192689_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_00_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:12,970 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:12,970 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] attempt: attempt_1463535192689_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:12,970 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:12,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000001, stoppedTime=1463535192970, exitStatus=0
2016-05-18 01:33:12,970 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B243B3E306934EDA844CBCFB2DC403CD, destinationVertexName=FDC9A2FF27E24C74964FCCD237F4FB29. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:12,971 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,971 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:12,971 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:12,972 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:12,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskId=task_1463535192689_0001_1_01_000000, scheduledTime=1463535192972, launchTime=1463535192972
2016-05-18 01:33:12,972 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000002, launchTime=1463535192975
2016-05-18 01:33:12,977 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000002
2016-05-18 01:33:12,978 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_01_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:33:12,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, startTime=1463535192978, containerId=container_1463535192689_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:33:12,978 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:12,978 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:12,978 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000001, Exiting
2016-05-18 01:33:12,979 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,979 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000001
2016-05-18 01:33:12,979 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,980 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:12 Starting to run new task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,980 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000001 completed successfully
2016-05-18 01:33:12,980 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,981 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:12,981 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:12,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: FDC9A2FF27E24C74964FCCD237F4FB29, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B243B3E306934EDA844CBCFB2DC403CD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5DFDE462F5E409084AA77724F0033C8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:33:12,982 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:12,982 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,984 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535192689_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:12,984 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:12,985 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:12,987 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:12,992 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, ordinal: 1
2016-05-18 01:33:12,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, mem on start (mb), free: 239, total: 393, max: 672
2016-05-18 01:33:12,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:12,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:12,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:12,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:12,993 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B243B3E306934EDA844CBCFB2DC403CD:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [A5DFDE462F5E409084AA77724F0033C8:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:33:12,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:12,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:12,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 316250512, TotalRequestedScaled: 1.58125256E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.45
2016-05-18 01:33:12,993 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B243B3E306934EDA844CBCFB2DC403CD:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [A5DFDE462F5E409084AA77724F0033C8:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:33:12,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:12,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B243B3E306934EDA844CBCFB2DC403CD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:12,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:12,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:12,994 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B243B3E306934EDA844CBCFB2DC403CD: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:12,995 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B243B3E306934EDA844CBCFB2DC403CD: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:12,995 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B243B3E306934EDA844CBCFB2DC403CD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:12,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:12,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:12,997 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:12,997 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B243B3E306934EDA844CBCFB2DC403CD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:12,999 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535192689_0001_1_00_000000_0_10081} to DISK_DIRECT, csize=85, dsize=81, EndTime=1463535192999, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 01:33:12,999 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,000 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,000 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B243B3E306934EDA844CBCFB2DC403CD, wasInterrupted=false
2016-05-18 01:33:13,001 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]]
2016-05-18 01:33:13,001 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,002 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:13,002 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] streamed: true, id: A3329097CFFB41DF974267568EB44BEF
2016-05-18 01:33:13,002 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]], id: 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,002 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_00_000000_0_10081/file.out, len=85, isLocal=true
2016-05-18 01:33:13,003 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 85 bytes from disk
2016-05-18 01:33:13,003 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:13,003 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:13,003 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,003 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 64 bytes
2016-05-18 01:33:13,003 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,003 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B243B3E306934EDA844CBCFB2DC403CD: Shuffle Runner thread complete
2016-05-18 01:33:13,004 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - B243B3E306934EDA844CBCFB2DC403CD: Already shutdown. Ignoring fetch complete
2016-05-18 01:33:13,005 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A5DFDE462F5E409084AA77724F0033C8 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@766e32f2, comparator=cascading.tuple.hadoop.util.TupleComparator@3b28c6f6, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:33:13,005 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A5DFDE462F5E409084AA77724F0033C8: , UsingHashComparator=false
2016-05-18 01:33:13,017 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:33:13,018 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:33:13,018 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A5DFDE462F5E409084AA77724F0033C8: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:33:13,018 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A5DFDE462F5E409084AA77724F0033C8: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:33:13,019 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] A3329097CFFB41DF974267568EB44BEF, for 1 inputs
2016-05-18 01:33:13,019 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:13,019 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B243B3E306934EDA844CBCFB2DC403CD: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:13,021 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,022 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDC9A2FF27E24C74964FCCD237F4FB29, mem on close (mb), free: 136, total: 393, max: 672
2016-05-18 01:33:13,022 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B243B3E306934EDA844CBCFB2DC403CD
2016-05-18 01:33:13,022 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B243B3E306934EDA844CBCFB2DC403CD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:13,022 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A5DFDE462F5E409084AA77724F0033C8: Starting flush of map output
2016-05-18 01:33:13,023 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A5DFDE462F5E409084AA77724F0033C8: Span0.length = 5, perItem = 13
2016-05-18 01:33:13,023 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A5DFDE462F5E409084AA77724F0033C8: done sorting span=0, length=5, time=0
2016-05-18 01:33:13,023 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A5DFDE462F5E409084AA77724F0033C8: Heap = SpanIterator<0:4> (span=Span[16777216,65]),
2016-05-18 01:33:13,024 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A5DFDE462F5E409084AA77724F0033C8: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_01_000000_0_10084_0/file.out
2016-05-18 01:33:13,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,028 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:33:13,028 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:33:13,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_01_000000_0: Counters: 40 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0]]
2016-05-18 01:33:13,029 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,029 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,030 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:13,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskAttemptId=attempt_1463535192689_0001_1_01_000000_0, creationTime=1463535192972, allocationTime=1463535192975, startTime=1463535192978, finishTime=1463535193030, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,030 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,030 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_01_000000_0
2016-05-18 01:33:13,031 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, taskId=task_1463535192689_0001_1_01_000000, startTime=1463535192978, finishTime=1463535193031, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_01_000000_0, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0
2016-05-18 01:33:13,032 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,032 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:13,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=FDC9A2FF27E24C74964FCCD237F4FB29, vertexId=vertex_1463535192689_0001_1_01, initRequestedTime=1463535192876, initedTime=1463535192881, startRequestedTime=1463535192895, startedTime=1463535192895, finishTime=1463535193032, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, OUTPUT_BYTES=65, OUTPUT_BYTES_WITH_OVERHEAD=81, OUTPUT_BYTES_PHYSICAL=85, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193019, Process_Duration=2, Process_End_Time=1463535193021, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, vertexStats=firstTaskStartTime=1463535192978, firstTasksToStart=[ task_1463535192689_0001_1_01_000000 ], lastTaskFinishTime=1463535193030, lastTasksToFinish=[ task_1463535192689_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_01_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,033 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000002, stoppedTime=1463535193033, exitStatus=0
2016-05-18 01:33:13,034 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] attempt: attempt_1463535192689_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:33:13,034 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A5DFDE462F5E409084AA77724F0033C8
2016-05-18 01:33:13,034 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, destinationVertexName=A5DFDE462F5E409084AA77724F0033C8. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:33:13,034 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,034 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:13,035 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,035 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_STARTED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskId=task_1463535192689_0001_1_02_000000, scheduledTime=1463535193035, launchTime=1463535193035
2016-05-18 01:33:13,035 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535192689_0001_00_000003, launchTime=1463535193036
2016-05-18 01:33:13,037 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,038 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535192689_0001_1_02_000000_0] started. Is using containerId: [container_1463535192689_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:33:13,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, startTime=1463535193038, containerId=container_1463535192689_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:33:13,038 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,039 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,039 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,039 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:13,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars, VertexName: A5DFDE462F5E409084AA77724F0033C8, VertexParallelism: 1, TaskAttemptID:attempt_1463535192689_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDC9A2FF27E24C74964FCCD237F4FB29, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=29A042B03F7B4091B3E9D134E2EE9553, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:13,039 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:13,040 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,040 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463535192689_0001_1_02_000000_0 1 events [0,1) total 1 vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,041 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535192689_0001_1_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:33:13,053 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,053 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000002: 22 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,054 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000002, Exiting
2016-05-18 01:33:13,054 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,054 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000002
2016-05-18 01:33:13,054 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,054 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000002 completed successfully
2016-05-18 01:33:13,055 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:13,059 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, ordinal: 2
2016-05-18 01:33:13,060 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, mem on start (mb), free: 346, total: 394, max: 672
2016-05-18 01:33:13,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:13,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:13,061 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:33:13,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:13,066 INFO output.MROutput (MROutput.java:initialize(421)) - 29A042B03F7B4091B3E9D134E2EE9553: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:13,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:13,066 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[29A042B03F7B4091B3E9D134E2EE9553:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [FDC9A2FF27E24C74964FCCD237F4FB29:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:33:13,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:13,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:13,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:33:13,067 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[29A042B03F7B4091B3E9D134E2EE9553:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [FDC9A2FF27E24C74964FCCD237F4FB29:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:33:13,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:13,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FDC9A2FF27E24C74964FCCD237F4FB29 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:13,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:13,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:13,071 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FDC9A2FF27E24C74964FCCD237F4FB29: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:33:13,071 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FDC9A2FF27E24C74964FCCD237F4FB29: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:33:13,072 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FDC9A2FF27E24C74964FCCD237F4FB29 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:33:13,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:13,074 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,075 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FDC9A2FF27E24C74964FCCD237F4FB29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:33:13,076 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463535192689_0001_1_01_000000_0_10084} to DISK_DIRECT, csize=85, dsize=81, EndTime=1463535193076, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 01:33:13,076 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,076 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,077 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FDC9A2FF27E24C74964FCCD237F4FB29, wasInterrupted=false
2016-05-18 01:33:13,077 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:33:13,078 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]]
2016-05-18 01:33:13,078 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] streamed: true, id: 9A60F89A455B4AE5A210FEEE435340A2
2016-05-18 01:33:13,078 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:33:13,078 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"], id: 29A042B03F7B4091B3E9D134E2EE9553
2016-05-18 01:33:13,079 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars"]
2016-05-18 01:33:13,079 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(**&%&%bar:bar@foo://blah/ (*(**^**&%&%^@#@&&() :::: ///\\ illegal chars in it)[by:[{1}:0]] 9A60F89A455B4AE5A210FEEE435340A2, for 1 inputs
2016-05-18 01:33:13,079 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001_wd/localmode-local-dir/output/attempt_1463535192689_0001_1_01_000000_0_10084/file.out, len=85, isLocal=true
2016-05-18 01:33:13,079 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 85 bytes from disk
2016-05-18 01:33:13,079 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:33:13,079 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:33:13,080 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 64 bytes
2016-05-18 01:33:13,080 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,080 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FDC9A2FF27E24C74964FCCD237F4FB29: Shuffle Runner thread complete
2016-05-18 01:33:13,080 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:33:13,080 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FDC9A2FF27E24C74964FCCD237F4FB29: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:33:13,083 INFO output.MROutput (MROutput.java:close(526)) - 29A042B03F7B4091B3E9D134E2EE9553 closed
2016-05-18 01:33:13,083 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535192689_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:33:13,084 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,084 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5DFDE462F5E409084AA77724F0033C8, mem on close (mb), free: 343, total: 394, max: 672
2016-05-18 01:33:13,084 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FDC9A2FF27E24C74964FCCD237F4FB29
2016-05-18 01:33:13,084 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FDC9A2FF27E24C74964FCCD237F4FB29: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:33:13,084 INFO output.MROutput (MROutput.java:close(526)) - 29A042B03F7B4091B3E9D134E2EE9553 closed
2016-05-18 01:33:13,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,085 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:33:13,085 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535192689_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:33:13,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535192689_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:33:13,087 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535192689_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,087 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskAttemptId=attempt_1463535192689_0001_1_02_000000_0, creationTime=1463535193035, allocationTime=1463535193036, startTime=1463535193038, finishTime=1463535193087, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,087 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535192689_0001_1_02_000000_0
2016-05-18 01:33:13,087 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,088 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:13,088 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,088 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,088 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:TASK_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, taskId=task_1463535192689_0001_1_02_000000, startTime=1463535193038, finishTime=1463535193087, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463535192689_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:33:13,089 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,089 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8]
2016-05-18 01:33:13,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:VERTEX_FINISHED]: vertexName=A5DFDE462F5E409084AA77724F0033C8, vertexId=vertex_1463535192689_0001_1_02, initRequestedTime=1463535192882, initedTime=1463535192888, startRequestedTime=1463535192896, startedTime=1463535192897, finishTime=1463535193089, timeTaken=192, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=85, SHUFFLE_BYTES=85, SHUFFLE_BYTES_DECOMPRESSED=81, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=85, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463535193080, Process_Duration=4, Process_End_Time=1463535193084, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463535193038, firstTasksToStart=[ task_1463535192689_0001_1_02_000000 ], lastTaskFinishTime=1463535193087, lastTasksToFinish=[ task_1463535192689_0001_1_02_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535192689_0001_1_02_000000 ], longestDurationTasks=[ task_1463535192689_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,090 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535192689_0001_00_000003, stoppedTime=1463535193090, exitStatus=0
2016-05-18 01:33:13,091 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:33:13,091 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,091 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535192689_0001_1_01 [FDC9A2FF27E24C74964FCCD237F4FB29]
2016-05-18 01:33:13,091 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463535192689_0001_1_00 [B243B3E306934EDA844CBCFB2DC403CD]
2016-05-18 01:33:13,091 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535192689_0001_1
2016-05-18 01:33:13,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535192689_0001_1
2016-05-18 01:33:13,092 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 29A042B03F7B4091B3E9D134E2EE9553 for vertex: vertex_1463535192689_0001_1_02 [A5DFDE462F5E409084AA77724F0033C8], outputName: 29A042B03F7B4091B3E9D134E2EE9553
2016-05-18 01:33:13,093 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:13,093 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535192689_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,099 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535192689_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,099 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535192689_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,099 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535192689_0001_00_000003, Exiting
2016-05-18 01:33:13,099 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,099 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535192689_0001_00_000003
2016-05-18 01:33:13,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535192689_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535192689_0001_00_000003 completed successfully
2016-05-18 01:33:13,101 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:29A042B03F7B4091B3E9D134E2EE9553 of vertex/vertexGroup:A5DFDE462F5E409084AA77724F0033C8 isVertexGroupOutput:false
2016-05-18 01:33:13,101 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,104 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535192689_0001_1, queueSize=0
2016-05-18 01:33:13,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535192689_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535192689_0001_1, startTime=1463535192874, finishTime=1463535193101, timeTaken=227, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=450, AM_GC_TIME_MILLIS=22, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=130, OUTPUT_BYTES_WITH_OVERHEAD=162, OUTPUT_BYTES_PHYSICAL=170, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=170, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=170, SHUFFLE_BYTES_DECOMPRESSED=162, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=170, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390605579056, Process_Duration=8, Process_End_Time=4390605579064, Read_Duration=0, Tuples_Read=15, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:33:13,105 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535192689_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:13,105 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535192689_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:13 Completed Dag: dag_1463535192689_0001_1
2016-05-18 01:33:13,105 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535192689_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:13,105 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535192689_0001_1_post
2016-05-18 01:33:13,105 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,105 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:13,106 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:13,106 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars, with id=dag_1463535192689_0001_1
2016-05-18 01:33:13,106 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D] **&%&%bar:bar@foo://blah/ (*(**^/(1/1) ...sInTempFiles/illegalchars, with id=dag_1463535192689_0001_1
2016-05-18 01:33:13,114 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sInTempFiles/illegalchars, applicationId=application_1463535192689_0001
2016-05-18 01:33:13,116 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:13,116 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:13,116 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:13,116 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:13,116 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:13,116 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:13,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:13,118 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:13,118 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:13,119 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,119 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:13,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:13,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,121 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:13,125 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/715785A88A184558986998991E340BA2/A9368079D89A4ED1996AFE5D554DA79D/.tez/application_1463535192689_0001
2016-05-18 01:33:13,126 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars/_temporary
2016-05-18 01:33:13,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [**&%&%bar:bar@foo://bl...] completed in: 00:00.459
2016-05-18 01:33:13,164 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testIllegalCharsInTempFiles/illegalchars
2016-05-18 01:33:13,190 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:33:13,190 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:33:13,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-18 01:33:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:33:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:13,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:33:13,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:33:13,243 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:33:13,243 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"]
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:33:13,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:33:13,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,254 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:33:13,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:33:13,272 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:13,273 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:33:13,273 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:33:13,282 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001 doesn't exist and is created
2016-05-18 01:33:13,301 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:33:13,301 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:33:13,302 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd
2016-05-18 01:33:13,317 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463535193273_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:33:13,317 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:13,317 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:33:13,319 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:33:13,319 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:33:13,320 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:33:13,320 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:33:13,321 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:33:13,322 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:33:13,322 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:33:13,323 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:33:13,323 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:33:13,323 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:33:13,323 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:33:13,323 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:33:13,324 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:33:13,324 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:13,324 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:33:13,324 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:33:13,324 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:33:13,326 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:33:13,327 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:33:13,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:33:13,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir
2016-05-18 01:33:13,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/history.txt.appattempt_1463535193273_0001_000000, maxErrors=10
2016-05-18 01:33:13,329 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:33:13,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463535193273_0001_000000, appSubmitTime=1463535193316, launchTime=1463535193316
2016-05-18 01:33:13,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:33:13,338 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46154
2016-05-18 01:33:13,339 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463535193273
2016-05-18 01:33:13,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463535193273_0001_000000, startTime=1463535193341
2016-05-18 01:33:13,341 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:33:13,401 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:33:13,401 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:33:13,402 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001, dagName=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,419 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,419 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:33:13,420 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463535193273_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/dag_1463535193273_0001_1.dot
2016-05-18 01:33:13,421 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001_wd/localmode-log-dir/dag_1463535193273_0001_1-tez-dag.pb.txt
2016-05-18 01:33:13,425 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535193273_0001_1
2016-05-18 01:33:13,425 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,426 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13 Running Dag: dag_1463535193273_0001_1
2016-05-18 01:33:13,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463535193273_0001_1, submitTime=1463535193419
2016-05-18 01:33:13,442 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001/tez-conf.pb" } size: 40938 timestamp: 1463535193000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:33:13,442 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,443 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,443 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:33:13,443 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:33:13,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463535193273_0001_1, initTime=1463535193444
2016-05-18 01:33:13,444 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:33:13,445 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:33:13,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_STARTED]: dagID=dag_1463535193273_0001_1, startTime=1463535193445
2016-05-18 01:33:13,445 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:33:13,445 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D5FB588337814B8BA6086D4A928CBB08 : {87E2105144FB4566BF81561E0E4E1D18={InputName=87E2105144FB4566BF81561E0E4E1D18}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:33:13,445 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 87E2105144FB4566BF81561E0E4E1D18, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:33:13,445 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,446 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:33:13,447 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 87E2105144FB4566BF81561E0E4E1D18 on vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,447 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001, dagName=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic
2016-05-18 01:33:13,450 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 87E2105144FB4566BF81561E0E4E1D18 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:33:13,452 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:33:13,452 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:33:13,453 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:33:13,453 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:33:13,453 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 87E2105144FB4566BF81561E0E4E1D18 on vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,456 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:13,456 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {87E2105144FB4566BF81561E0E4E1D18=forAllWorkUnits=true, update=[1]}
2016-05-18 01:33:13,456 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] parallelism set to 1
2016-05-18 01:33:13,456 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:33:13,457 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], numAdditionalOutputs=1
2016-05-18 01:33:13,457 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C52A942D71F2402D9AE4485BCA5014B8, vertex=vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:33:13,458 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D5FB588337814B8BA6086D4A928CBB08:C52A942D71F2402D9AE4485BCA5014B8 using oldmapred API
2016-05-18 01:33:13,458 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C52A942D71F2402D9AE4485BCA5014B8, vertexName=D5FB588337814B8BA6086D4A928CBB08, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:33:13,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, vertexId=vertex_1463535193273_0001_1_00, initRequestedTime=1463535193445, initedTime=1463535193457, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:33:13,460 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] with distanceFromRoot: 0
2016-05-18 01:33:13,460 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:33:13,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463535193273_0001_1_00, startRequestedTime=1463535193447, startedTime=1463535193460
2016-05-18 01:33:13,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463535193273_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:33:13,461 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:33:13,461 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D5FB588337814B8BA6086D4A928CBB08
2016-05-18 01:33:13,461 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_STARTED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskId=task_1463535193273_0001_1_00_000000, scheduledTime=1463535193462, launchTime=1463535193462
2016-05-18 01:33:13,462 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,463 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:33:13,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463535193273_0001_00_000001, launchTime=1463535193464
2016-05-18 01:33:13,465 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,466 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535193273_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,466 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463535193273_0001_1_00_000000_0] started. Is using containerId: [container_1463535193273_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:33:13,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, startTime=1463535193466, containerId=container_1463535193273_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:33:13,467 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463535193273_0001, with dag id: dag_1463535193273_0001_1
2016-05-18 01:33:13 Starting to run new task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,467 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:33:13,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, VertexName: D5FB588337814B8BA6086D4A928CBB08, VertexParallelism: 1, TaskAttemptID:attempt_1463535193273_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=87E2105144FB4566BF81561E0E4E1D18, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C52A942D71F2402D9AE4485BCA5014B8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:33:13,468 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:33:13,469 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,470 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463535193273_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:33:13,470 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463535193273_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:33:13,473 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:33:13,475 INFO input.MRInput (MRInput.java:initialize(440)) - 87E2105144FB4566BF81561E0E4E1D18 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:33:13,475 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,480 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, ordinal: 0
2016-05-18 01:33:13,481 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, mem on start (mb), free: 266, total: 394, max: 672
2016-05-18 01:33:13,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:33:13,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:33:13,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:33:13,485 INFO output.MROutput (MROutput.java:initialize(421)) - C52A942D71F2402D9AE4485BCA5014B8: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:33:13,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:33:13,486 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[87E2105144FB4566BF81561E0E4E1D18:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [C52A942D71F2402D9AE4485BCA5014B8:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:33:13,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:33:13,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:33:13,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:33:13,488 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[87E2105144FB4566BF81561E0E4E1D18:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [C52A942D71F2402D9AE4485BCA5014B8:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:33:13,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:33:13,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 87E2105144FB4566BF81561E0E4E1D18 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:33:13,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:33:13,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:33:13,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:33:13,492 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,494 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:33:13,495 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 87E2105144FB4566BF81561E0E4E1D18 initialized RecordReader from event
2016-05-18 01:33:13,495 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,496 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: 87E2105144FB4566BF81561E0E4E1D18
2016-05-18 01:33:13,496 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"], id: C52A942D71F2402D9AE4485BCA5014B8
2016-05-18 01:33:13,496 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic"]
2016-05-18 01:33:13,497 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:33:13,497 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:33:13,499 INFO output.MROutput (MROutput.java:close(526)) - C52A942D71F2402D9AE4485BCA5014B8 closed
2016-05-18 01:33:13,500 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463535193273_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:33:13,500 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,500 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5FB588337814B8BA6086D4A928CBB08, mem on close (mb), free: 264, total: 394, max: 672
2016-05-18 01:33:13,501 INFO output.MROutput (MROutput.java:close(526)) - C52A942D71F2402D9AE4485BCA5014B8 closed
2016-05-18 01:33:13,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:33:13 Completed running task attempt: attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,502 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:33:13,502 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463535193273_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:33:13,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463535193273_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-18 01:33:13,503 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463535193273_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskAttemptId=attempt_1463535193273_0001_1_00_000000_0, creationTime=1463535193462, allocationTime=1463535193464, startTime=1463535193466, finishTime=1463535193503, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:33:13,503 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:33:13,504 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:33:13,504 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463535193273_0001_1_00_000000_0
2016-05-18 01:33:13,504 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:TASK_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, taskId=task_1463535193273_0001_1_00_000000, startTime=1463535193466, finishTime=1463535193504, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463535193273_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:33:13,504 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,505 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:33:13,505 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08]
2016-05-18 01:33:13,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:VERTEX_FINISHED]: vertexName=D5FB588337814B8BA6086D4A928CBB08, vertexId=vertex_1463535193273_0001_1_00, initRequestedTime=1463535193445, initedTime=1463535193457, startRequestedTime=1463535193447, startedTime=1463535193460, finishTime=1463535193505, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463535193466, firstTasksToStart=[ task_1463535193273_0001_1_00_000000 ], lastTaskFinishTime=1463535193503, lastTasksToFinish=[ task_1463535193273_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463535193273_0001_1_00_000000 ], longestDurationTasks=[ task_1463535193273_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:33:13,506 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:33:13,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463535193273_0001_00_000001, stoppedTime=1463535193506, exitStatus=0
2016-05-18 01:33:13,506 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:33:13,506 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,507 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463535193273_0001_1
2016-05-18 01:33:13,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463535193273_0001_1
2016-05-18 01:33:13,507 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C52A942D71F2402D9AE4485BCA5014B8 for vertex: vertex_1463535193273_0001_1_00 [D5FB588337814B8BA6086D4A928CBB08], outputName: C52A942D71F2402D9AE4485BCA5014B8
2016-05-18 01:33:13,510 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:33:13,510 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463535193273_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:33:13,515 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463535193273_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:33:13,515 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463535193273_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:33:13,516 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463535193273_0001_00_000001, Exiting
2016-05-18 01:33:13,516 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,516 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463535193273_0001_00_000001
2016-05-18 01:33:13,516 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463535193273_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:33:13,517 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463535193273_0001_00_000001 completed successfully
2016-05-18 01:33:13,519 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C52A942D71F2402D9AE4485BCA5014B8 of vertex/vertexGroup:D5FB588337814B8BA6086D4A928CBB08 isVertexGroupOutput:false
2016-05-18 01:33:13,519 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:33:13,520 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463535193273_0001_1, queueSize=0
2016-05-18 01:33:13,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463535193273_0001_1][Event:DAG_FINISHED]: dagId=dag_1463535193273_0001_1, startTime=1463535193445, finishTime=1463535193519, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463535193497, Process_Duration=3, Process_End_Time=1463535193500, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:33:13,520 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463535193273_0001_1 finished with state: SUCCEEDED
2016-05-18 01:33:13,521 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463535193273_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:33:13 Completed Dag: dag_1463535193273_0001_1
2016-05-18 01:33:13,521 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463535193273_0001_1, dagState=SUCCEEDED
2016-05-18 01:33:13,521 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463535193273_0001_1_post
2016-05-18 01:33:13,521 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:33:13,521 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:33:13,521 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:33:13,521 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, with id=dag_1463535193273_0001_1
2016-05-18 01:33:13,521 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD] test/(1/1) ...-unknown-nondeterministic, with id=dag_1463535193273_0001_1
2016-05-18 01:33:13,528 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...-unknown-nondeterministic, applicationId=application_1463535193273_0001
2016-05-18 01:33:13,529 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:33:13,529 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:33:13,530 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:33:13,530 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:33:13,530 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:33:13,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:33:13,531 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:33:13,531 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:33:13,531 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:33:13,532 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,532 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:33:13,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:33:13,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:33:13,534 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:33:13,536 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B7DF1531F74F4A8887A6E22DB43D0EB5/F33279DF76A741E9B6F1DDC57469CBAD/.tez/application_1463535193273_0001
2016-05-18 01:33:13,537 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic/_temporary
2016-05-18 01:33:13,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.284
2016-05-18 01:33:13,572 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/regressionpipesplatform/testUnGroupUnknown/ungrouped-unknown-nondeterministic