2016-05-18 01:21:09,929 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 01:21:09,959 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-05-18 01:21:09,994 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 01:21:10,555 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 01:21:10,879 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 01:21:11,144 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:11,146 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:11,242 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 71D5B21DA0CC49D29138C7D69CCB4D24
2016-05-18 01:21:11,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.131
2016-05-18 01:21:11,547 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.157
2016-05-18 01:21:11,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:11,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:11,550 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:11,719 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:11,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-18 01:21:12,188 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 01:21:12,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:12,192 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:12,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-18 01:21:12,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:12,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:12,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:12,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-05-18 01:21:12,195 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:12,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:12,391 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:12,399 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:12,399 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:12,472 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001 doesn't exist and is created
2016-05-18 01:21:12,545 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:12,546 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:12,547 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd
2016-05-18 01:21:12,612 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534472398_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:12,626 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:12,627 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:12,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:12,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:12,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:12,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:12,646 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 01:21:12,646 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 01:21:12,646 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:12,662 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:12,685 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:12,686 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:12,688 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:12,689 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:12,689 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:12,690 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:12,691 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:12,692 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:12,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:12,693 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:12,705 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:12,710 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:12,732 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:12,735 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:12,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:12,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-log-dir
2016-05-18 01:21:12,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-log-dir/history.txt.appattempt_1463534472398_0001_000000, maxErrors=10
2016-05-18 01:21:12,741 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:12,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534472398_0001_000000, appSubmitTime=1463534472590, launchTime=1463534472610
2016-05-18 01:21:12,747 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 01:21:12,747 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 01:21:12,747 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:12,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:12,803 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38675
2016-05-18 01:21:12,805 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534472398
2016-05-18 01:21:12,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534472398_0001_000000, startTime=1463534472820
2016-05-18 01:21:12,821 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:12,848 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:12,863 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:12,863 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463534472398_0001, dagName=[58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain
2016-05-18 01:21:13,097 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain
2016-05-18 01:21:13,106 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:13,169 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534472398_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-log-dir/dag_1463534472398_0001_1.dot
2016-05-18 01:21:13,173 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-log-dir/dag_1463534472398_0001_1-tez-dag.pb.txt
2016-05-18 01:21:13,219 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534472398_0001_1
2016-05-18 01:21:13,219 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:13,220 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain
2016-05-18 01:21:13 Running Dag: dag_1463534472398_0001_1
2016-05-18 01:21:13,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534472398_0001_1, submitTime=1463534473098
2016-05-18 01:21:13,303 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001/tez-conf.pb" } size: 40949 timestamp: 1463534472000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:13,304 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,305 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:13,307 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001/tez-conf.pb" } size: 40949 timestamp: 1463534472000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:13,308 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:13,309 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001/tez-conf.pb" } size: 40949 timestamp: 1463534472000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:13,310 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:13,311 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:13,320 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:13,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534472398_0001_1, initTime=1463534473322
2016-05-18 01:21:13,324 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534472398_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:13,325 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:13,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:DAG_STARTED]: dagID=dag_1463534472398_0001_1, startTime=1463534473327
2016-05-18 01:21:13,328 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534472398_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:13,328 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BEA0D046F521437099C2975C80A6C706 : {E21B7FF5EA86440CA2CAE740BF9629A6={InputName=E21B7FF5EA86440CA2CAE740BF9629A6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:13,329 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: E21B7FF5EA86440CA2CAE740BF9629A6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:13,329 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,338 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,338 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,340 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463534472398_0001, dagName=[58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain
2016-05-18 01:21:13,341 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,348 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: E21B7FF5EA86440CA2CAE740BF9629A6 on vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,360 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:13,361 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:13,368 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input E21B7FF5EA86440CA2CAE740BF9629A6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:13,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534472398_0001, with dag id: dag_1463534472398_0001_1
2016-05-18 01:21:13,408 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:13,409 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:13,419 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:13,419 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:13,421 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:13,421 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:13,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:13,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534472398_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:13,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E186C997FB924161B590A78706614C3A, vertexId=vertex_1463534472398_0001_1_01, initRequestedTime=1463534473360, initedTime=1463534473425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:13,427 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:13,429 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:13,441 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: E21B7FF5EA86440CA2CAE740BF9629A6 on vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,459 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:13,459 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:13,460 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:13,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534472398_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:13,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4], numAdditionalOutputs=1
2016-05-18 01:21:13,462 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A81E8EBE224A4E8AB29067BA25B79C2B, vertex=vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:13,473 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DFC7D7EFD0D642249E6565D587DA11C4:A81E8EBE224A4E8AB29067BA25B79C2B using oldmapred API
2016-05-18 01:21:13,475 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A81E8EBE224A4E8AB29067BA25B79C2B, vertexName=DFC7D7EFD0D642249E6565D587DA11C4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:13,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, vertexId=vertex_1463534472398_0001_1_02, initRequestedTime=1463534473429, initedTime=1463534473461, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:13,485 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:13,491 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:13,492 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {E21B7FF5EA86440CA2CAE740BF9629A6=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:13,493 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] parallelism set to 1
2016-05-18 01:21:13,493 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:13,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEA0D046F521437099C2975C80A6C706, vertexId=vertex_1463534472398_0001_1_00, initRequestedTime=1463534473328, initedTime=1463534473502, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:13,503 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] with distanceFromRoot: 0
2016-05-18 01:21:13,503 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:13,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534472398_0001_1_00, startRequestedTime=1463534473360, startedTime=1463534473503
2016-05-18 01:21:13,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534472398_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:13,505 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:13,505 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534472398_0001_1_00 for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] numStartedSources: 1 numSources: 1
2016-05-18 01:21:13,506 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] with distanceFromRoot: 1
2016-05-18 01:21:13,508 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BEA0D046F521437099C2975C80A6C706
2016-05-18 01:21:13,508 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534472398_0001_1_01, startRequestedTime=1463534473505, startedTime=1463534473506
2016-05-18 01:21:13,519 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:13,519 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534472398_0001_1_01 for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] numStartedSources: 1 numSources: 1
2016-05-18 01:21:13,520 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] with distanceFromRoot: 2
2016-05-18 01:21:13,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534472398_0001_1_02, startRequestedTime=1463534473519, startedTime=1463534473520
2016-05-18 01:21:13,524 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:13,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_STARTED]: vertexName=BEA0D046F521437099C2975C80A6C706, taskId=task_1463534472398_0001_1_00_000000, scheduledTime=1463534473534, launchTime=1463534473534
2016-05-18 01:21:13,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: DFC7D7EFD0D642249E6565D587DA11C4 with 0 source tasks and 1 pending tasks
2016-05-18 01:21:13,541 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: E186C997FB924161B590A78706614C3A with 0 source tasks and 1 pending tasks
2016-05-18 01:21:13,545 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E186C997FB924161B590A78706614C3A in vertex: DFC7D7EFD0D642249E6565D587DA11C4 numBipartiteSourceTasks: 1
2016-05-18 01:21:13,551 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: BEA0D046F521437099C2975C80A6C706 in vertex: E186C997FB924161B590A78706614C3A numBipartiteSourceTasks: 1
2016-05-18 01:21:13,575 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:13,617 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:13,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534472398_0001_00_000001, launchTime=1463534473632
2016-05-18 01:21:13,638 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000001
2016-05-18 01:21:13,641 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:13,644 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534472398_0001_1_00_000000_0] started. Is using containerId: [container_1463534472398_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:13 Starting to run new task attempt: attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:13,649 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:13,649 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:13,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEA0D046F521437099C2975C80A6C706, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0, startTime=1463534473642, containerId=container_1463534472398_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:13,651 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:13,651 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 01:21:13,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain, VertexName: BEA0D046F521437099C2975C80A6C706, VertexParallelism: 1, TaskAttemptID:attempt_1463534472398_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E21B7FF5EA86440CA2CAE740BF9629A6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E186C997FB924161B590A78706614C3A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:13,663 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:13,667 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:13,675 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534472398_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:13,679 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534472398_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:13,704 INFO input.MRInput (MRInput.java:initialize(440)) - E21B7FF5EA86440CA2CAE740BF9629A6 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:13,706 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: E21B7FF5EA86440CA2CAE740BF9629A6
2016-05-18 01:21:13,712 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:13,726 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEA0D046F521437099C2975C80A6C706, ordinal: 0
2016-05-18 01:21:13,726 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEA0D046F521437099C2975C80A6C706, mem on start (mb), free: 197, total: 281, max: 672
2016-05-18 01:21:13,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:13,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:13,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:13,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:13,728 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[E21B7FF5EA86440CA2CAE740BF9629A6:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [E186C997FB924161B590A78706614C3A:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:13,734 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:13,740 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:13,741 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:13,741 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[E21B7FF5EA86440CA2CAE740BF9629A6:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [E186C997FB924161B590A78706614C3A:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:13,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:13,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E21B7FF5EA86440CA2CAE740BF9629A6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:13,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E21B7FF5EA86440CA2CAE740BF9629A6
2016-05-18 01:21:13,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:13,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:13,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:13,746 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:13,773 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:13,779 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - E21B7FF5EA86440CA2CAE740BF9629A6 initialized RecordReader from event
2016-05-18 01:21:13,788 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:13,824 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: E21B7FF5EA86440CA2CAE740BF9629A6
2016-05-18 01:21:13,824 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 0171F1AF25FC48B6B8F2D4BF4F9513DD
2016-05-18 01:21:13,825 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 0171F1AF25FC48B6B8F2D4BF4F9513DD
2016-05-18 01:21:13,875 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - E186C997FB924161B590A78706614C3A using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@7c711591, comparator=cascading.tuple.hadoop.util.TupleComparator@1f0715ce, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:13,881 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for E186C997FB924161B590A78706614C3A: , UsingHashComparator=false
2016-05-18 01:21:13,994 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:13,995 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:13,995 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for E186C997FB924161B590A78706614C3A: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:13,998 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - E186C997FB924161B590A78706614C3A: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:14,004 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:14,006 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEA0D046F521437099C2975C80A6C706, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:21:14,015 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:14,015 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEA0D046F521437099C2975C80A6C706, mem on close (mb), free: 84, total: 281, max: 672
2016-05-18 01:21:14,016 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - E186C997FB924161B590A78706614C3A: Starting flush of map output
2016-05-18 01:21:14,016 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - E186C997FB924161B590A78706614C3A: Span0.length = 10, perItem = 21
2016-05-18 01:21:14,018 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - E186C997FB924161B590A78706614C3A: done sorting span=0, length=10, time=2
2016-05-18 01:21:14,021 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - E186C997FB924161B590A78706614C3A: Heap = SpanIterator<0:9> (span=Span[16777216,210]),
2016-05-18 01:21:14,027 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - E186C997FB924161B590A78706614C3A: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-local-dir/output/attempt_1463534472398_0001_1_00_000000_0_10002_0/file.out
2016-05-18 01:21:14,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:14 Completed running task attempt: attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:14,067 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:14,067 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534472398_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:14,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534472398_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534474006, Process_Duration=9, Process_End_Time=1463534474015, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:21:14,077 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534472398_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,078 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,078 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:14,078 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,079 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000001
2016-05-18 01:21:14,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEA0D046F521437099C2975C80A6C706, taskAttemptId=attempt_1463534472398_0001_1_00_000000_0, creationTime=1463534473566, allocationTime=1463534473625, startTime=1463534473642, finishTime=1463534474076, timeTaken=434, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:14,090 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534472398_0001_1_00_000000_0
2016-05-18 01:21:14,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_FINISHED]: vertexName=BEA0D046F521437099C2975C80A6C706, taskId=task_1463534472398_0001_1_00_000000, startTime=1463534473642, finishTime=1463534474092, timeTaken=450, status=SUCCEEDED, successfulAttemptID=attempt_1463534472398_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534474006, Process_Duration=9, Process_End_Time=1463534474015, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:21:14,109 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:14,109 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:14,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_FINISHED]: vertexName=BEA0D046F521437099C2975C80A6C706, vertexId=vertex_1463534472398_0001_1_00, initRequestedTime=1463534473328, initedTime=1463534473502, startRequestedTime=1463534473360, startedTime=1463534473503, finishTime=1463534474109, timeTaken=606, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534474006, Process_Duration=9, Process_End_Time=1463534474015, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463534473642, firstTasksToStart=[ task_1463534472398_0001_1_00_000000 ], lastTaskFinishTime=1463534474076, lastTasksToFinish=[ task_1463534472398_0001_1_00_000000 ], minTaskDuration=434, maxTaskDuration=434, avgTaskDuration=434.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534472398_0001_1_00_000000 ], longestDurationTasks=[ task_1463534472398_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:14,122 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:14,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534472398_0001_00_000001, stoppedTime=1463534474123, exitStatus=0
2016-05-18 01:21:14,124 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] attempt: attempt_1463534472398_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:14,124 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:14,125 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:14,125 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534472398_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:14,125 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E186C997FB924161B590A78706614C3A
2016-05-18 01:21:14,129 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=BEA0D046F521437099C2975C80A6C706, destinationVertexName=E186C997FB924161B590A78706614C3A. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:14,129 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:14,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_STARTED]: vertexName=E186C997FB924161B590A78706614C3A, taskId=task_1463534472398_0001_1_01_000000, scheduledTime=1463534474131, launchTime=1463534474131
2016-05-18 01:21:14,132 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:14,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534472398_0001_00_000002, launchTime=1463534474134
2016-05-18 01:21:14,135 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000002
2016-05-18 01:21:14,136 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534472398_0001_1_01_000000_0] started. Is using containerId: [container_1463534472398_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:14,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E186C997FB924161B590A78706614C3A, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0, startTime=1463534474136, containerId=container_1463534472398_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:14,137 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14 Starting to run new task attempt: attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,138 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,139 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:14,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain, VertexName: E186C997FB924161B590A78706614C3A, VertexParallelism: 1, TaskAttemptID:attempt_1463534472398_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEA0D046F521437099C2975C80A6C706, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DFC7D7EFD0D642249E6565D587DA11C4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:14,140 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:14,140 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,143 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534472398_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:14,143 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000001: 62 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:14,143 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534472398_0001_00_000001, Exiting
2016-05-18 01:21:14,143 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534472398_0001_00_000001
2016-05-18 01:21:14,143 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534472398_0001_00_000001
2016-05-18 01:21:14,152 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:14,156 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534472398_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:14,157 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534472398_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:14,158 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534472398_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:14,158 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534472398_0001_00_000001 completed successfully
2016-05-18 01:21:14,167 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:14,172 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E186C997FB924161B590A78706614C3A, ordinal: 1
2016-05-18 01:21:14,173 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E186C997FB924161B590A78706614C3A, mem on start (mb), free: 71, total: 281, max: 672
2016-05-18 01:21:14,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:14,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:14,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:14,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:14,174 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DFC7D7EFD0D642249E6565D587DA11C4:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput], [BEA0D046F521437099C2975C80A6C706:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:14,175 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:14,175 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:14,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 316250512, TotalRequestedScaled: 1.58125256E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.45
2016-05-18 01:21:14,176 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DFC7D7EFD0D642249E6565D587DA11C4:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600], [BEA0D046F521437099C2975C80A6C706:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:14,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:14,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BEA0D046F521437099C2975C80A6C706 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:14,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:14,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:14,179 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - BEA0D046F521437099C2975C80A6C706: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:14,183 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - BEA0D046F521437099C2975C80A6C706: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:14,203 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: BEA0D046F521437099C2975C80A6C706 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:14,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BEA0D046F521437099C2975C80A6C706
2016-05-18 01:21:14,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:14,209 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,218 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-18 01:21:14,221 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BEA0D046F521437099C2975C80A6C706: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:14,225 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 0171F1AF25FC48B6B8F2D4BF4F9513DD
2016-05-18 01:21:14,226 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 56EA859D77814F48BB93DE5173FDD3E6
2016-05-18 01:21:14,226 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 56EA859D77814F48BB93DE5173FDD3E6
2016-05-18 01:21:14,231 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - DFC7D7EFD0D642249E6565D587DA11C4 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@11ff6c15, comparator=cascading.tuple.hadoop.util.TupleComparator@1810dfee, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:14,232 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for DFC7D7EFD0D642249E6565D587DA11C4: , UsingHashComparator=false
2016-05-18 01:21:14,335 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:14,335 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534472398_0001_1_00_000000_0_10002} to DISK_DIRECT, csize=204, dsize=200, EndTime=1463534474335, TimeTaken=4, Rate=0.05 MB/s
2016-05-18 01:21:14,335 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:14,336 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for DFC7D7EFD0D642249E6565D587DA11C4: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:14,336 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : BEA0D046F521437099C2975C80A6C706
2016-05-18 01:21:14,337 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - DFC7D7EFD0D642249E6565D587DA11C4: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:14,338 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:14,338 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 0171F1AF25FC48B6B8F2D4BF4F9513DD, for 1 inputs
2016-05-18 01:21:14,338 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: BEA0D046F521437099C2975C80A6C706, wasInterrupted=false
2016-05-18 01:21:14,339 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:14,341 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - BEA0D046F521437099C2975C80A6C706: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:14,343 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:14,344 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-local-dir/output/attempt_1463534472398_0001_1_00_000000_0_10002/file.out, len=204, isLocal=true
2016-05-18 01:21:14,345 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 204 bytes from disk
2016-05-18 01:21:14,346 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:14,350 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:14,354 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-18 01:21:14,355 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : BEA0D046F521437099C2975C80A6C706
2016-05-18 01:21:14,355 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E186C997FB924161B590A78706614C3A, all 1 inputs ready in: 00:00:00.017
2016-05-18 01:21:14,356 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - BEA0D046F521437099C2975C80A6C706: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:14,355 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - BEA0D046F521437099C2975C80A6C706: Shuffle Runner thread complete
2016-05-18 01:21:14,369 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,369 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E186C997FB924161B590A78706614C3A, mem on close (mb), free: 64, total: 382, max: 672
2016-05-18 01:21:14,369 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: BEA0D046F521437099C2975C80A6C706
2016-05-18 01:21:14,370 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BEA0D046F521437099C2975C80A6C706: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:14,370 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - DFC7D7EFD0D642249E6565D587DA11C4: Starting flush of map output
2016-05-18 01:21:14,370 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - DFC7D7EFD0D642249E6565D587DA11C4: Span0.length = 8, perItem = 23
2016-05-18 01:21:14,371 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - DFC7D7EFD0D642249E6565D587DA11C4: done sorting span=0, length=8, time=1
2016-05-18 01:21:14,371 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - DFC7D7EFD0D642249E6565D587DA11C4: Heap = SpanIterator<0:7> (span=Span[16777216,184]),
2016-05-18 01:21:14,372 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - DFC7D7EFD0D642249E6565D587DA11C4: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-local-dir/output/attempt_1463534472398_0001_1_01_000000_0_10006_0/file.out
2016-05-18 01:21:14,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:14 Completed running task attempt: attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,377 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:14,378 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534472398_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:14,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534472398_0001_1_01_000000_0: Counters: 40 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=160, MERGE_PHASE_TIME=175, FIRST_EVENT_RECEIVED=42, LAST_EVENT_RECEIVED=42][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534474355, Process_Duration=14, Process_End_Time=1463534474369, Read_Duration=4, Tuples_Read=10, Tuples_Written=8, Write_Duration=0]]
2016-05-18 01:21:14,386 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534472398_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,386 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E186C997FB924161B590A78706614C3A, taskAttemptId=attempt_1463534472398_0001_1_01_000000_0, creationTime=1463534474132, allocationTime=1463534474133, startTime=1463534474136, finishTime=1463534474386, timeTaken=250, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:14,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:14,387 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,387 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000002
2016-05-18 01:21:14,388 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534472398_0001_1_01_000000_0
2016-05-18 01:21:14,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_FINISHED]: vertexName=E186C997FB924161B590A78706614C3A, taskId=task_1463534472398_0001_1_01_000000, startTime=1463534474136, finishTime=1463534474388, timeTaken=252, status=SUCCEEDED, successfulAttemptID=attempt_1463534472398_0001_1_01_000000_0, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=160, MERGE_PHASE_TIME=175, FIRST_EVENT_RECEIVED=42, LAST_EVENT_RECEIVED=42, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534474355, Process_Duration=14, Process_End_Time=1463534474369, Read_Duration=4, Tuples_Read=10, Tuples_Written=8, Write_Duration=0
2016-05-18 01:21:14,391 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:14,392 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:14,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_FINISHED]: vertexName=E186C997FB924161B590A78706614C3A, vertexId=vertex_1463534472398_0001_1_01, initRequestedTime=1463534473360, initedTime=1463534473425, startRequestedTime=1463534473505, startedTime=1463534473506, finishTime=1463534474392, timeTaken=886, status=SUCCEEDED, diagnostics=, counters=Counters: 40, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=160, MERGE_PHASE_TIME=175, FIRST_EVENT_RECEIVED=42, LAST_EVENT_RECEIVED=42, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534474355, Process_Duration=14, Process_End_Time=1463534474369, Read_Duration=4, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, vertexStats=firstTaskStartTime=1463534474136, firstTasksToStart=[ task_1463534472398_0001_1_01_000000 ], lastTaskFinishTime=1463534474386, lastTasksToFinish=[ task_1463534472398_0001_1_01_000000 ], minTaskDuration=250, maxTaskDuration=250, avgTaskDuration=250.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534472398_0001_1_01_000000 ], longestDurationTasks=[ task_1463534472398_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:14,398 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:14,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534472398_0001_00_000002, stoppedTime=1463534474398, exitStatus=0
2016-05-18 01:21:14,399 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] attempt: attempt_1463534472398_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:14,399 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:14,399 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DFC7D7EFD0D642249E6565D587DA11C4
2016-05-18 01:21:14,400 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E186C997FB924161B590A78706614C3A, destinationVertexName=DFC7D7EFD0D642249E6565D587DA11C4. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:14,400 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:14,399 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:14,400 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534472398_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:14,403 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000002: 15 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:14,403 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534472398_0001_00_000002, Exiting
2016-05-18 01:21:14,402 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534472398_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:14,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_STARTED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, taskId=task_1463534472398_0001_1_02_000000, scheduledTime=1463534474403, launchTime=1463534474403
2016-05-18 01:21:14,404 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534472398_0001_00_000002
2016-05-18 01:21:14,405 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534472398_0001_00_000002
2016-05-18 01:21:14,405 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:14,405 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534472398_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:14,406 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534472398_0001_00_000002 completed successfully
2016-05-18 01:21:14,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534472398_0001_00_000003, launchTime=1463534474409
2016-05-18 01:21:14,409 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000003
2016-05-18 01:21:14,410 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534472398_0001_1_02_000000_0] started. Is using containerId: [container_1463534472398_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:21:14,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0, startTime=1463534474410, containerId=container_1463534472398_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:21:14,411 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14 Starting to run new task attempt: attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,412 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,413 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,414 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:14,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain, VertexName: DFC7D7EFD0D642249E6565D587DA11C4, VertexParallelism: 1, TaskAttemptID:attempt_1463534472398_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E186C997FB924161B590A78706614C3A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A81E8EBE224A4E8AB29067BA25B79C2B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:14,414 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:14,416 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,418 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534472398_0001_1_02_000000_0 1 events [0,1) total 1 vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:14,418 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534472398_0001_1_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:14,422 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:14,425 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:14,437 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DFC7D7EFD0D642249E6565D587DA11C4, ordinal: 2
2016-05-18 01:21:14,437 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFC7D7EFD0D642249E6565D587DA11C4, mem on start (mb), free: 59, total: 382, max: 672
2016-05-18 01:21:14,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:14,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:14,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:14,449 INFO output.MROutput (MROutput.java:initialize(421)) - A81E8EBE224A4E8AB29067BA25B79C2B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:14,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:14,451 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A81E8EBE224A4E8AB29067BA25B79C2B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E186C997FB924161B590A78706614C3A:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:14,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:14,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:14,452 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:14,452 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A81E8EBE224A4E8AB29067BA25B79C2B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E186C997FB924161B590A78706614C3A:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:14,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:14,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E186C997FB924161B590A78706614C3A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:14,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:14,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:14,455 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E186C997FB924161B590A78706614C3A: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:14,455 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E186C997FB924161B590A78706614C3A: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:14,456 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E186C997FB924161B590A78706614C3A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:14,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E186C997FB924161B590A78706614C3A
2016-05-18 01:21:14,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:14,458 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,460 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E186C997FB924161B590A78706614C3A: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:14,466 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-18 01:21:14,467 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534472398_0001_1_01_000000_0_10006} to DISK_DIRECT, csize=210, dsize=206, EndTime=1463534474467, TimeTaken=3, Rate=0.07 MB/s
2016-05-18 01:21:14,468 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 56EA859D77814F48BB93DE5173FDD3E6
2016-05-18 01:21:14,469 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: A81E8EBE224A4E8AB29067BA25B79C2B
2016-05-18 01:21:14,468 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E186C997FB924161B590A78706614C3A
2016-05-18 01:21:14,469 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-05-18 01:21:14,470 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:14,471 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E186C997FB924161B590A78706614C3A, wasInterrupted=false
2016-05-18 01:21:14,471 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 56EA859D77814F48BB93DE5173FDD3E6, for 1 inputs
2016-05-18 01:21:14,471 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:14,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:14,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001_wd/localmode-local-dir/output/attempt_1463534472398_0001_1_01_000000_0_10006/file.out, len=210, isLocal=true
2016-05-18 01:21:14,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 210 bytes from disk
2016-05-18 01:21:14,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:14,475 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:14,476 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-05-18 01:21:14,477 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E186C997FB924161B590A78706614C3A
2016-05-18 01:21:14,478 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E186C997FB924161B590A78706614C3A: Shuffle Runner thread complete
2016-05-18 01:21:14,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - E186C997FB924161B590A78706614C3A: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:14,478 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DFC7D7EFD0D642249E6565D587DA11C4, all 1 inputs ready in: 00:00:00.007
2016-05-18 01:21:14,480 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E186C997FB924161B590A78706614C3A: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:14,486 INFO output.MROutput (MROutput.java:close(526)) - A81E8EBE224A4E8AB29067BA25B79C2B closed
2016-05-18 01:21:14,487 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534472398_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:21:14,488 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,489 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFC7D7EFD0D642249E6565D587DA11C4, mem on close (mb), free: 55, total: 382, max: 672
2016-05-18 01:21:14,489 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E186C997FB924161B590A78706614C3A
2016-05-18 01:21:14,489 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E186C997FB924161B590A78706614C3A: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:14,489 INFO output.MROutput (MROutput.java:close(526)) - A81E8EBE224A4E8AB29067BA25B79C2B closed
2016-05-18 01:21:14,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:14 Completed running task attempt: attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,491 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:21:14,492 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534472398_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:21:14,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534472398_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534474480, Process_Duration=8, Process_End_Time=1463534474488, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-18 01:21:14,495 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534472398_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, taskAttemptId=attempt_1463534472398_0001_1_02_000000_0, creationTime=1463534474405, allocationTime=1463534474408, startTime=1463534474410, finishTime=1463534474495, timeTaken=85, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:14,495 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:14,498 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:14,498 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,498 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534472398_0001_1_02_000000_0
2016-05-18 01:21:14,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:TASK_FINISHED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, taskId=task_1463534472398_0001_1_02_000000, startTime=1463534474410, finishTime=1463534474499, timeTaken=89, status=SUCCEEDED, successfulAttemptID=attempt_1463534472398_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534474480, Process_Duration=8, Process_End_Time=1463534474488, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-18 01:21:14,501 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:14,501 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4]
2016-05-18 01:21:14,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:VERTEX_FINISHED]: vertexName=DFC7D7EFD0D642249E6565D587DA11C4, vertexId=vertex_1463534472398_0001_1_02, initRequestedTime=1463534473429, initedTime=1463534473461, startRequestedTime=1463534473519, startedTime=1463534473520, finishTime=1463534474501, timeTaken=981, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534474480, Process_Duration=8, Process_End_Time=1463534474488, Read_Duration=2, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463534474410, firstTasksToStart=[ task_1463534472398_0001_1_02_000000 ], lastTaskFinishTime=1463534474495, lastTasksToFinish=[ task_1463534472398_0001_1_02_000000 ], minTaskDuration=85, maxTaskDuration=85, avgTaskDuration=85.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534472398_0001_1_02_000000 ], longestDurationTasks=[ task_1463534472398_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:14,503 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:14,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534472398_0001_00_000003, stoppedTime=1463534474503, exitStatus=0
2016-05-18 01:21:14,506 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:14,506 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:14,506 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534472398_0001_1_00 [BEA0D046F521437099C2975C80A6C706]
2016-05-18 01:21:14,507 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534472398_0001_1_01 [E186C997FB924161B590A78706614C3A]
2016-05-18 01:21:14,509 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534472398_0001_1
2016-05-18 01:21:14,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534472398_0001_1
2016-05-18 01:21:14,505 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534472398_0001_00_000003
2016-05-18 01:21:14,516 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534472398_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:21:14,516 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534472398_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:14,517 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534472398_0001_00_000003, Exiting
2016-05-18 01:21:14,517 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534472398_0001_00_000003
2016-05-18 01:21:14,517 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534472398_0001_00_000003
2016-05-18 01:21:14,517 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A81E8EBE224A4E8AB29067BA25B79C2B for vertex: vertex_1463534472398_0001_1_02 [DFC7D7EFD0D642249E6565D587DA11C4], outputName: A81E8EBE224A4E8AB29067BA25B79C2B
2016-05-18 01:21:14,517 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534472398_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:14,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534472398_0001_00_000003 completed successfully
2016-05-18 01:21:14,517 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534472398_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:14,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463534472398_0001_00_000003
2016-05-18 01:21:14,537 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A81E8EBE224A4E8AB29067BA25B79C2B of vertex/vertexGroup:DFC7D7EFD0D642249E6565D587DA11C4 isVertexGroupOutput:false
2016-05-18 01:21:14,537 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:14,546 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534472398_0001_1, queueSize=0
2016-05-18 01:21:14,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534472398_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534472398_0001_1, startTime=1463534473325, finishTime=1463534474537, timeTaken=1212, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=2810, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=176, MERGE_PHASE_TIME=197, FIRST_EVENT_RECEIVED=47, LAST_EVENT_RECEIVED=47, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390603422841, Process_Duration=31, Process_End_Time=4390603422872, Read_Duration=8, Tuples_Read=28, Tuples_Written=26, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-18 01:21:14,549 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534472398_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:14,549 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534472398_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:14 Completed Dag: dag_1463534472398_0001_1
2016-05-18 01:21:14,551 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534472398_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:14,552 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534472398_0001_1_post
2016-05-18 01:21:14,553 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:14,553 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:14,553 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:14,554 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain, with id=dag_1463534472398_0001_1
2016-05-18 01:21:14,593 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B] test/(1/1) ...stSimpleChain/simplechain, with id=dag_1463534472398_0001_1
2016-05-18 01:21:14,604 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1463534472398_0001
2016-05-18 01:21:14,612 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:14,612 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:14,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:14,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:14,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:14,615 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:14,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:14,617 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:14,617 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:14,617 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:14,618 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:14,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:14,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:14,620 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:14,623 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/58F5177B86ED4AC09EFF3AFE4A1CD95A/C84E780E2D6B4C2CBDE44D83D67D565B/.tez/application_1463534472398_0001
2016-05-18 01:21:14,625 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain/_temporary
2016-05-18 01:21:14,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:02.433
2016-05-18 01:21:14,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-05-18 01:21:14,721 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:14,721 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:14,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-05-18 01:21:14,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.052
2016-05-18 01:21:14,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:14,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:14,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:14,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 01:21:14,852 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-05-18 01:21:14,921 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-18 01:21:14,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:14,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-18 01:21:14,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-18 01:21:14,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-18 01:21:14,923 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-18 01:21:14,924 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-05-18 01:21:14,924 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:14,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-18 01:21:14,946 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:14,946 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:14,947 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:14,957 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001 doesn't exist and is created
2016-05-18 01:21:14,982 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:14,983 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:14,984 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd
2016-05-18 01:21:15,002 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534474946_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:15,003 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:15,003 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:15,005 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:15,005 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:15,006 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:15,006 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:15,007 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:15,011 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:15,011 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:15,011 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:15,012 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:15,012 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:15,012 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:15,013 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:15,013 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:15,013 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:15,013 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:15,014 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:15,014 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:15,019 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:15,020 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:15,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:15,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-log-dir
2016-05-18 01:21:15,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-log-dir/history.txt.appattempt_1463534474946_0001_000000, maxErrors=10
2016-05-18 01:21:15,022 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:15,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534474946_0001_000000, appSubmitTime=1463534475002, launchTime=1463534475002
2016-05-18 01:21:15,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:15,042 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35304
2016-05-18 01:21:15,043 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534474946
2016-05-18 01:21:15,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534474946_0001_000000, startTime=1463534475048
2016-05-18 01:21:15,048 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:15,083 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:15,083 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:15,083 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463534474946_0001, dagName=[4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count
2016-05-18 01:21:15,109 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count
2016-05-18 01:21:15,109 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:15,110 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534474946_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-log-dir/dag_1463534474946_0001_1.dot
2016-05-18 01:21:15,111 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-log-dir/dag_1463534474946_0001_1-tez-dag.pb.txt
2016-05-18 01:21:15,119 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534474946_0001_1
2016-05-18 01:21:15,120 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,120 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count
2016-05-18 01:21:15 Running Dag: dag_1463534474946_0001_1
2016-05-18 01:21:15,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534474946_0001_1, submitTime=1463534475109
2016-05-18 01:21:15,138 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001/tez-conf.pb" } size: 40949 timestamp: 1463534474000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:15,138 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,139 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:15,140 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001/tez-conf.pb" } size: 40949 timestamp: 1463534474000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:15,140 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,140 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:15,141 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:15,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534474946_0001_1, initTime=1463534475141
2016-05-18 01:21:15,142 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534474946_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:15,142 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:15,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:DAG_STARTED]: dagID=dag_1463534474946_0001_1, startTime=1463534475143
2016-05-18 01:21:15,143 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463534474946_0001, dagName=[4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count
2016-05-18 01:21:15,143 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534474946_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:15,143 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D12B938F8EC443DF80B746CEBDACB308 : {B29E4DFC183849229C48CCD7697105C1={InputName=B29E4DFC183849229C48CCD7697105C1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:15,143 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B29E4DFC183849229C48CCD7697105C1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:15,143 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,144 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,144 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,144 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,145 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:15,145 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,147 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B29E4DFC183849229C48CCD7697105C1 on vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,150 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:15,150 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,150 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534474946_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:15,152 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA], numAdditionalOutputs=1
2016-05-18 01:21:15,153 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=6F7A95C921FE40E6AD8BF93BD0912D03, vertex=vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:15,155 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B29E4DFC183849229C48CCD7697105C1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:15,158 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EDCB339AA3E44EA19232AF28FDB357BA:6F7A95C921FE40E6AD8BF93BD0912D03 using oldmapred API
2016-05-18 01:21:15,158 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=6F7A95C921FE40E6AD8BF93BD0912D03, vertexName=EDCB339AA3E44EA19232AF28FDB357BA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:15,159 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:15,159 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 01:21:15,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, vertexId=vertex_1463534474946_0001_1_01, initRequestedTime=1463534475145, initedTime=1463534475152, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:15,160 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:15,160 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:15,161 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:15,161 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B29E4DFC183849229C48CCD7697105C1 on vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,163 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:15,163 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B29E4DFC183849229C48CCD7697105C1=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:15,164 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] parallelism set to 1
2016-05-18 01:21:15,164 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:15,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, vertexId=vertex_1463534474946_0001_1_00, initRequestedTime=1463534475143, initedTime=1463534475165, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:15,166 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] with distanceFromRoot: 0
2016-05-18 01:21:15,166 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:15,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534474946_0001_1_00, startRequestedTime=1463534475145, startedTime=1463534475166
2016-05-18 01:21:15,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534474946_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:15,167 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:15,168 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534474946_0001_1_00 for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] numStartedSources: 1 numSources: 1
2016-05-18 01:21:15,168 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] with distanceFromRoot: 1
2016-05-18 01:21:15,168 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D12B938F8EC443DF80B746CEBDACB308
2016-05-18 01:21:15,169 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534474946_0001_1_01, startRequestedTime=1463534475168, startedTime=1463534475169
2016-05-18 01:21:15,172 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:15,172 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: EDCB339AA3E44EA19232AF28FDB357BA with 0 source tasks and 1 pending tasks
2016-05-18 01:21:15,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_STARTED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, taskId=task_1463534474946_0001_1_00_000000, scheduledTime=1463534475172, launchTime=1463534475172
2016-05-18 01:21:15,173 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463534474946_0001, with dag id: dag_1463534474946_0001_1
2016-05-18 01:21:15,174 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D12B938F8EC443DF80B746CEBDACB308 in vertex: EDCB339AA3E44EA19232AF28FDB357BA numBipartiteSourceTasks: 1
2016-05-18 01:21:15,176 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:15,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534474946_0001_00_000001, launchTime=1463534475178
2016-05-18 01:21:15,182 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534474946_0001_00_000001
2016-05-18 01:21:15,184 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534474946_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,184 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534474946_0001_1_00_000000_0] started. Is using containerId: [container_1463534474946_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:15 Starting to run new task attempt: attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0, startTime=1463534475183, containerId=container_1463534474946_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:15,187 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,187 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,188 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:15,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count, VertexName: D12B938F8EC443DF80B746CEBDACB308, VertexParallelism: 1, TaskAttemptID:attempt_1463534474946_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B29E4DFC183849229C48CCD7697105C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EDCB339AA3E44EA19232AF28FDB357BA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:15,189 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:15,190 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,192 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534474946_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:15,192 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534474946_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:15,199 INFO input.MRInput (MRInput.java:initialize(440)) - B29E4DFC183849229C48CCD7697105C1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:15,200 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B29E4DFC183849229C48CCD7697105C1
2016-05-18 01:21:15,201 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:15,210 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D12B938F8EC443DF80B746CEBDACB308, ordinal: 0
2016-05-18 01:21:15,210 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D12B938F8EC443DF80B746CEBDACB308, mem on start (mb), free: 54, total: 382, max: 672
2016-05-18 01:21:15,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:15,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:15,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:15,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:15,212 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B29E4DFC183849229C48CCD7697105C1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EDCB339AA3E44EA19232AF28FDB357BA:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:15,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:15,213 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:15,213 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:15,213 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B29E4DFC183849229C48CCD7697105C1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EDCB339AA3E44EA19232AF28FDB357BA:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:15,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:15,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B29E4DFC183849229C48CCD7697105C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:15,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:15,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:15,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B29E4DFC183849229C48CCD7697105C1
2016-05-18 01:21:15,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:15,217 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,223 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 01:21:15,224 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B29E4DFC183849229C48CCD7697105C1 initialized RecordReader from event
2016-05-18 01:21:15,226 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:15,226 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: B29E4DFC183849229C48CCD7697105C1
2016-05-18 01:21:15,226 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 8B66EF351B674A7693E4C6C338538070
2016-05-18 01:21:15,227 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 8B66EF351B674A7693E4C6C338538070
2016-05-18 01:21:15,229 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - EDCB339AA3E44EA19232AF28FDB357BA using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@50f0ab91, comparator=cascading.tuple.hadoop.util.TupleComparator@688fe61a, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:15,230 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for EDCB339AA3E44EA19232AF28FDB357BA: , UsingHashComparator=false
2016-05-18 01:21:15,339 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:15,340 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:15,340 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for EDCB339AA3E44EA19232AF28FDB357BA: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:15,340 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - EDCB339AA3E44EA19232AF28FDB357BA: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:15,341 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:15,341 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D12B938F8EC443DF80B746CEBDACB308, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:15,344 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,344 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D12B938F8EC443DF80B746CEBDACB308, mem on close (mb), free: 51, total: 482, max: 672
2016-05-18 01:21:15,345 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - EDCB339AA3E44EA19232AF28FDB357BA: Starting flush of map output
2016-05-18 01:21:15,345 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - EDCB339AA3E44EA19232AF28FDB357BA: Span0.length = 20, perItem = 23
2016-05-18 01:21:15,346 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - EDCB339AA3E44EA19232AF28FDB357BA: done sorting span=0, length=20, time=1
2016-05-18 01:21:15,347 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - EDCB339AA3E44EA19232AF28FDB357BA: Heap = SpanIterator<0:19> (span=Span[16777216,466]),
2016-05-18 01:21:15,348 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - EDCB339AA3E44EA19232AF28FDB357BA: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-local-dir/output/attempt_1463534474946_0001_1_00_000000_0_10012_0/file.out
2016-05-18 01:21:15,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:15 Completed running task attempt: attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,355 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:15,358 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534474946_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:15,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534474946_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534475341, Process_Duration=3, Process_End_Time=1463534475344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 01:21:15,360 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534474946_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:15,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, taskAttemptId=attempt_1463534474946_0001_1_00_000000_0, creationTime=1463534475173, allocationTime=1463534475177, startTime=1463534475183, finishTime=1463534475360, timeTaken=177, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:15,360 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:15,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:15,362 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,363 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534474946_0001_1_00_000000_0
2016-05-18 01:21:15,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_FINISHED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, taskId=task_1463534474946_0001_1_00_000000, startTime=1463534475183, finishTime=1463534475363, timeTaken=180, status=SUCCEEDED, successfulAttemptID=attempt_1463534474946_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534475341, Process_Duration=3, Process_End_Time=1463534475344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 01:21:15,364 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:15,364 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_FINISHED]: vertexName=D12B938F8EC443DF80B746CEBDACB308, vertexId=vertex_1463534474946_0001_1_00, initRequestedTime=1463534475143, initedTime=1463534475165, startRequestedTime=1463534475145, startedTime=1463534475166, finishTime=1463534475364, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534475341, Process_Duration=3, Process_End_Time=1463534475344, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463534475183, firstTasksToStart=[ task_1463534474946_0001_1_00_000000 ], lastTaskFinishTime=1463534475360, lastTasksToFinish=[ task_1463534474946_0001_1_00_000000 ], minTaskDuration=177, maxTaskDuration=177, avgTaskDuration=177.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534474946_0001_1_00_000000 ], longestDurationTasks=[ task_1463534474946_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:15,368 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:15,370 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] attempt: attempt_1463534474946_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:15,370 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EDCB339AA3E44EA19232AF28FDB357BA
2016-05-18 01:21:15,369 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534474946_0001_00_000001
2016-05-18 01:21:15,371 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534474946_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:15,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534474946_0001_00_000001, stoppedTime=1463534475370, exitStatus=0
2016-05-18 01:21:15,371 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D12B938F8EC443DF80B746CEBDACB308, destinationVertexName=EDCB339AA3E44EA19232AF28FDB357BA. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:15,371 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534474946_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:15,372 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,371 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:15,373 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534474946_0001_00_000001, Exiting
2016-05-18 01:21:15,373 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534474946_0001_00_000001
2016-05-18 01:21:15,373 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534474946_0001_00_000001
2016-05-18 01:21:15,374 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:15,374 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534474946_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:15,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_STARTED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, taskId=task_1463534474946_0001_1_01_000000, scheduledTime=1463534475374, launchTime=1463534475374
2016-05-18 01:21:15,375 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,377 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534474946_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:15,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534474946_0001_00_000001 completed successfully
2016-05-18 01:21:15,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534474946_0001_00_000002, launchTime=1463534475379
2016-05-18 01:21:15,380 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534474946_0001_00_000002
2016-05-18 01:21:15,381 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534474946_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,381 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534474946_0001_1_01_000000_0] started. Is using containerId: [container_1463534474946_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:15,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0, startTime=1463534475381, containerId=container_1463534474946_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:15 Starting to run new task attempt: attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,383 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,383 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,384 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:15,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count, VertexName: EDCB339AA3E44EA19232AF28FDB357BA, VertexParallelism: 1, TaskAttemptID:attempt_1463534474946_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D12B938F8EC443DF80B746CEBDACB308, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6F7A95C921FE40E6AD8BF93BD0912D03, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:15,385 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:15,385 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,388 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534474946_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,388 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534474946_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:15,394 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:15,397 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:15,412 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EDCB339AA3E44EA19232AF28FDB357BA, ordinal: 1
2016-05-18 01:21:15,412 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDCB339AA3E44EA19232AF28FDB357BA, mem on start (mb), free: 45, total: 482, max: 672
2016-05-18 01:21:15,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:15,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:15,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:15,448 INFO output.MROutput (MROutput.java:initialize(421)) - 6F7A95C921FE40E6AD8BF93BD0912D03: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:15,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:15,451 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[D12B938F8EC443DF80B746CEBDACB308:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [6F7A95C921FE40E6AD8BF93BD0912D03:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:15,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:15,453 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:15,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:15,455 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[D12B938F8EC443DF80B746CEBDACB308:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [6F7A95C921FE40E6AD8BF93BD0912D03:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:15,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:15,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D12B938F8EC443DF80B746CEBDACB308 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:15,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:15,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:15,457 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D12B938F8EC443DF80B746CEBDACB308: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:15,457 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D12B938F8EC443DF80B746CEBDACB308: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:15,458 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D12B938F8EC443DF80B746CEBDACB308 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:15,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D12B938F8EC443DF80B746CEBDACB308
2016-05-18 01:21:15,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:15,461 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,463 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D12B938F8EC443DF80B746CEBDACB308: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:15,465 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-18 01:21:15,466 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534474946_0001_1_00_000000_0_10012} to DISK_DIRECT, csize=463, dsize=459, EndTime=1463534475466, TimeTaken=2, Rate=0.22 MB/s
2016-05-18 01:21:15,466 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 8B66EF351B674A7693E4C6C338538070
2016-05-18 01:21:15,467 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 6F7A95C921FE40E6AD8BF93BD0912D03
2016-05-18 01:21:15,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D12B938F8EC443DF80B746CEBDACB308
2016-05-18 01:21:15,467 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-05-18 01:21:15,468 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 8B66EF351B674A7693E4C6C338538070, for 1 inputs
2016-05-18 01:21:15,468 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:15,469 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D12B938F8EC443DF80B746CEBDACB308, wasInterrupted=false
2016-05-18 01:21:15,470 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:15,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:15,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001_wd/localmode-local-dir/output/attempt_1463534474946_0001_1_00_000000_0_10012/file.out, len=463, isLocal=true
2016-05-18 01:21:15,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 463 bytes from disk
2016-05-18 01:21:15,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:15,472 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:15,472 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-18 01:21:15,473 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D12B938F8EC443DF80B746CEBDACB308
2016-05-18 01:21:15,473 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D12B938F8EC443DF80B746CEBDACB308: Shuffle Runner thread complete
2016-05-18 01:21:15,473 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EDCB339AA3E44EA19232AF28FDB357BA, all 1 inputs ready in: 00:00:00.005
2016-05-18 01:21:15,474 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D12B938F8EC443DF80B746CEBDACB308: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:15,479 INFO output.MROutput (MROutput.java:close(526)) - 6F7A95C921FE40E6AD8BF93BD0912D03 closed
2016-05-18 01:21:15,480 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534474946_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:15,481 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,481 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDCB339AA3E44EA19232AF28FDB357BA, mem on close (mb), free: 259, total: 589, max: 672
2016-05-18 01:21:15,481 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D12B938F8EC443DF80B746CEBDACB308
2016-05-18 01:21:15,482 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D12B938F8EC443DF80B746CEBDACB308: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:15,482 INFO output.MROutput (MROutput.java:close(526)) - 6F7A95C921FE40E6AD8BF93BD0912D03 closed
2016-05-18 01:21:15,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:15 Completed running task attempt: attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,483 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:15,484 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534474946_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:15,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534474946_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534475473, Process_Duration=8, Process_End_Time=1463534475481, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-18 01:21:15,486 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534474946_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:15,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, taskAttemptId=attempt_1463534474946_0001_1_01_000000_0, creationTime=1463534475375, allocationTime=1463534475377, startTime=1463534475381, finishTime=1463534475486, timeTaken=105, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:15,487 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:15,488 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534474946_0001_1_01_000000_0
2016-05-18 01:21:15,488 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:15,489 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,489 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534474946_0001_00_000002
2016-05-18 01:21:15,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:TASK_FINISHED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, taskId=task_1463534474946_0001_1_01_000000, startTime=1463534475381, finishTime=1463534475488, timeTaken=107, status=SUCCEEDED, successfulAttemptID=attempt_1463534474946_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534475473, Process_Duration=8, Process_End_Time=1463534475481, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-18 01:21:15,490 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:15,490 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA]
2016-05-18 01:21:15,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:VERTEX_FINISHED]: vertexName=EDCB339AA3E44EA19232AF28FDB357BA, vertexId=vertex_1463534474946_0001_1_01, initRequestedTime=1463534475145, initedTime=1463534475152, startRequestedTime=1463534475168, startedTime=1463534475169, finishTime=1463534475490, timeTaken=321, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534475473, Process_Duration=8, Process_End_Time=1463534475481, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463534475381, firstTasksToStart=[ task_1463534474946_0001_1_01_000000 ], lastTaskFinishTime=1463534475486, lastTasksToFinish=[ task_1463534474946_0001_1_01_000000 ], minTaskDuration=105, maxTaskDuration=105, avgTaskDuration=105.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534474946_0001_1_01_000000 ], longestDurationTasks=[ task_1463534474946_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:15,491 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:15,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534474946_0001_00_000002, stoppedTime=1463534475492, exitStatus=0
2016-05-18 01:21:15,492 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:15,492 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:15,492 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534474946_0001_1_00 [D12B938F8EC443DF80B746CEBDACB308]
2016-05-18 01:21:15,492 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534474946_0001_1
2016-05-18 01:21:15,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534474946_0001_1
2016-05-18 01:21:15,493 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 6F7A95C921FE40E6AD8BF93BD0912D03 for vertex: vertex_1463534474946_0001_1_01 [EDCB339AA3E44EA19232AF28FDB357BA], outputName: 6F7A95C921FE40E6AD8BF93BD0912D03
2016-05-18 01:21:15,493 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534474946_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:15,495 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534474946_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:15,500 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534474946_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:15,500 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534474946_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:15,500 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534474946_0001_00_000002, Exiting
2016-05-18 01:21:15,500 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534474946_0001_00_000002
2016-05-18 01:21:15,501 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534474946_0001_00_000002
2016-05-18 01:21:15,503 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534474946_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:15,503 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534474946_0001_00_000002 completed successfully
2016-05-18 01:21:15,508 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:6F7A95C921FE40E6AD8BF93BD0912D03 of vertex/vertexGroup:EDCB339AA3E44EA19232AF28FDB357BA isVertexGroupOutput:false
2016-05-18 01:21:15,508 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:15,510 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534474946_0001_1, queueSize=0
2016-05-18 01:21:15,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534474946_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534474946_0001_1, startTime=1463534475142, finishTime=1463534475508, timeTaken=366, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=790, AM_GC_TIME_MILLIS=30, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068950814, Process_Duration=11, Process_End_Time=2927068950825, Read_Duration=5, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-18 01:21:15,511 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534474946_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:15,511 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534474946_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:15 Completed Dag: dag_1463534474946_0001_1
2016-05-18 01:21:15,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534474946_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:15,511 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534474946_0001_1_post
2016-05-18 01:21:15,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:15,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:15,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count, with id=dag_1463534474946_0001_1
2016-05-18 01:21:15,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6] count/(1/1) ...splatform/testCount/count, with id=dag_1463534474946_0001_1
2016-05-18 01:21:15,525 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1463534474946_0001
2016-05-18 01:21:15,528 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:15,528 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:15,528 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:15,528 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:15,528 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:15,529 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:15,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:15,530 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:15,530 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:15,530 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:15,530 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:15,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:15,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:15,533 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:15,536 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/4E9957F2E8DC4F9AB521F50487A61674/A8A49AE09C8F43599CABF057656545A6/.tez/application_1463534474946_0001
2016-05-18 01:21:15,537 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count/_temporary
2016-05-18 01:21:15,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.615
2016-05-18 01:21:15,628 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-05-18 01:21:15,660 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:15,661 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:15,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-18 01:21:15,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-18 01:21:15,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:15,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:15,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:15,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:15,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-18 01:21:15,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:15,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:15,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-18 01:21:15,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:15,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:15,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:15,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-05-18 01:21:15,766 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:15,784 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:15,789 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:15,789 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:15,790 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:15,799 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001 doesn't exist and is created
2016-05-18 01:21:15,823 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:15,823 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:15,824 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001_wd
2016-05-18 01:21:15,840 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534475789_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:15,840 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:15,840 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:15,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:15,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:15,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:15,842 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:15,843 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:15,845 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:15,845 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:15,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:15,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:15,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:15,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:15,847 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:15,849 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:15,850 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:15,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:15,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001_wd/localmode-log-dir
2016-05-18 01:21:15,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001_wd/localmode-log-dir/history.txt.appattempt_1463534475789_0001_000000, maxErrors=10
2016-05-18 01:21:15,851 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:15,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534475789_0001_000000, appSubmitTime=1463534475839, launchTime=1463534475839
2016-05-18 01:21:15,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:15,864 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49329
2016-05-18 01:21:15,865 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534475789
2016-05-18 01:21:15,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534475789_0001_000000, startTime=1463534475870
2016-05-18 01:21:15,870 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:15,923 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:15,923 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:15,924 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463534475789_0001, dagName=[20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap
2016-05-18 01:21:15,943 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap
2016-05-18 01:21:15,943 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:15,944 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534475789_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001_wd/localmode-log-dir/dag_1463534475789_0001_1.dot
2016-05-18 01:21:15,945 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001_wd/localmode-log-dir/dag_1463534475789_0001_1-tez-dag.pb.txt
2016-05-18 01:21:15,950 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534475789_0001_1
2016-05-18 01:21:15,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:15,950 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap
2016-05-18 01:21:15 Running Dag: dag_1463534475789_0001_1
2016-05-18 01:21:15,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534475789_0001_1, submitTime=1463534475943
2016-05-18 01:21:15,966 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001/tez-conf.pb" } size: 40949 timestamp: 1463534475000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:15,966 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,966 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,966 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:15,967 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:15,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534475789_0001_1, initTime=1463534475968
2016-05-18 01:21:15,969 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534475789_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:15,969 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:15,970 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463534475789_0001, dagName=[20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap
2016-05-18 01:21:15,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:DAG_STARTED]: dagID=dag_1463534475789_0001_1, startTime=1463534475970
2016-05-18 01:21:15,970 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534475789_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:15,970 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BAF3578B26124890A87F4BBBC7BD4EF2 : {21C26336EF304ABC8CD1BD8472B93496={InputName=21C26336EF304ABC8CD1BD8472B93496}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:15,970 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 21C26336EF304ABC8CD1BD8472B93496, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:15,971 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,971 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,971 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,971 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,972 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:15,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 21C26336EF304ABC8CD1BD8472B93496 on vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,975 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 21C26336EF304ABC8CD1BD8472B93496 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:15,978 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:15,979 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:15,980 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 21C26336EF304ABC8CD1BD8472B93496 on vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,980 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:15,981 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {21C26336EF304ABC8CD1BD8472B93496=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:15,982 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] parallelism set to 1
2016-05-18 01:21:15,982 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:15,982 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2], numAdditionalOutputs=1
2016-05-18 01:21:15,983 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D1DAA64622724BED8C1F2A2E7A2F1728, vertex=vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:15,985 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BAF3578B26124890A87F4BBBC7BD4EF2:D1DAA64622724BED8C1F2A2E7A2F1728 using oldmapred API
2016-05-18 01:21:15,985 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D1DAA64622724BED8C1F2A2E7A2F1728, vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:15,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, vertexId=vertex_1463534475789_0001_1_00, initRequestedTime=1463534475970, initedTime=1463534475982, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:15,988 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] with distanceFromRoot: 0
2016-05-18 01:21:15,988 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:15,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534475789_0001_1_00, startRequestedTime=1463534475972, startedTime=1463534475988
2016-05-18 01:21:15,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534475789_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:15,989 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:15,989 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BAF3578B26124890A87F4BBBC7BD4EF2
2016-05-18 01:21:15,990 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:TASK_STARTED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, taskId=task_1463534475789_0001_1_00_000000, scheduledTime=1463534475990, launchTime=1463534475990
2016-05-18 01:21:15,991 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:15,993 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:15,997 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534475789_0001, with dag id: dag_1463534475789_0001_1
2016-05-18 01:21:15,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534475789_0001_00_000001, launchTime=1463534475997
2016-05-18 01:21:15,998 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534475789_0001_00_000001
2016-05-18 01:21:15,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534475789_0001_1_00_000000_0] started. Is using containerId: [container_1463534475789_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:16,000 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534475789_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0, startTime=1463534475999, containerId=container_1463534475789_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:16 Starting to run new task attempt: attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,002 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,002 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,002 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:16,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap, VertexName: BAF3578B26124890A87F4BBBC7BD4EF2, VertexParallelism: 1, TaskAttemptID:attempt_1463534475789_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=21C26336EF304ABC8CD1BD8472B93496, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D1DAA64622724BED8C1F2A2E7A2F1728, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:16,003 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:16,004 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534475789_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:16,004 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,005 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534475789_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:16,009 INFO input.MRInput (MRInput.java:initialize(440)) - 21C26336EF304ABC8CD1BD8472B93496 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:16,009 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 21C26336EF304ABC8CD1BD8472B93496
2016-05-18 01:21:16,011 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:16,019 INFO output.MROutput (MROutput.java:initialize(421)) - D1DAA64622724BED8C1F2A2E7A2F1728: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:16,020 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BAF3578B26124890A87F4BBBC7BD4EF2, ordinal: 0
2016-05-18 01:21:16,020 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BAF3578B26124890A87F4BBBC7BD4EF2, mem on start (mb), free: 168, total: 589, max: 672
2016-05-18 01:21:16,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:16,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:16,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:16,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:16,021 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[21C26336EF304ABC8CD1BD8472B93496:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D1DAA64622724BED8C1F2A2E7A2F1728:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:16,021 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:16,021 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:16,022 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:21:16,022 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[21C26336EF304ABC8CD1BD8472B93496:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D1DAA64622724BED8C1F2A2E7A2F1728:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:16,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:16,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 21C26336EF304ABC8CD1BD8472B93496 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:16,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 21C26336EF304ABC8CD1BD8472B93496
2016-05-18 01:21:16,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:16,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:16,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:16,023 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,024 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:16,025 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 21C26336EF304ABC8CD1BD8472B93496 initialized RecordReader from event
2016-05-18 01:21:16,027 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:16,028 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 21C26336EF304ABC8CD1BD8472B93496
2016-05-18 01:21:16,028 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: D1DAA64622724BED8C1F2A2E7A2F1728
2016-05-18 01:21:16,028 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-05-18 01:21:16,029 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:16,030 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BAF3578B26124890A87F4BBBC7BD4EF2, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:16,032 INFO output.MROutput (MROutput.java:close(526)) - D1DAA64622724BED8C1F2A2E7A2F1728 closed
2016-05-18 01:21:16,033 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534475789_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:21:16,034 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,034 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BAF3578B26124890A87F4BBBC7BD4EF2, mem on close (mb), free: 167, total: 589, max: 672
2016-05-18 01:21:16,035 INFO output.MROutput (MROutput.java:close(526)) - D1DAA64622724BED8C1F2A2E7A2F1728 closed
2016-05-18 01:21:16,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:16 Completed running task attempt: attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,036 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:16,036 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534475789_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:16,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534475789_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463534476030, Process_Duration=4, Process_End_Time=1463534476034, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-18 01:21:16,037 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534475789_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:16,038 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:16,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, taskAttemptId=attempt_1463534475789_0001_1_00_000000_0, creationTime=1463534475991, allocationTime=1463534475996, startTime=1463534475999, finishTime=1463534476037, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:16,039 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534475789_0001_1_00_000000_0
2016-05-18 01:21:16,038 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:16,040 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:TASK_FINISHED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, taskId=task_1463534475789_0001_1_00_000000, startTime=1463534475999, finishTime=1463534476039, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463534475789_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476030, Process_Duration=4, Process_End_Time=1463534476034, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:21:16,040 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534475789_0001_00_000001
2016-05-18 01:21:16,041 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:16,042 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2]
2016-05-18 01:21:16,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:VERTEX_FINISHED]: vertexName=BAF3578B26124890A87F4BBBC7BD4EF2, vertexId=vertex_1463534475789_0001_1_00, initRequestedTime=1463534475970, initedTime=1463534475982, startRequestedTime=1463534475972, startedTime=1463534475988, finishTime=1463534476042, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476030, Process_Duration=4, Process_End_Time=1463534476034, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463534475999, firstTasksToStart=[ task_1463534475789_0001_1_00_000000 ], lastTaskFinishTime=1463534476037, lastTasksToFinish=[ task_1463534475789_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534475789_0001_1_00_000000 ], longestDurationTasks=[ task_1463534475789_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:16,044 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:16,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534475789_0001_00_000001, stoppedTime=1463534476044, exitStatus=0
2016-05-18 01:21:16,044 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:21:16,044 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:16,044 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534475789_0001_1
2016-05-18 01:21:16,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534475789_0001_1
2016-05-18 01:21:16,045 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534475789_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:16,045 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D1DAA64622724BED8C1F2A2E7A2F1728 for vertex: vertex_1463534475789_0001_1_00 [BAF3578B26124890A87F4BBBC7BD4EF2], outputName: D1DAA64622724BED8C1F2A2E7A2F1728
2016-05-18 01:21:16,045 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534475789_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:16,054 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534475789_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:16,054 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534475789_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:16,054 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534475789_0001_00_000001, Exiting
2016-05-18 01:21:16,054 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534475789_0001_00_000001
2016-05-18 01:21:16,054 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534475789_0001_00_000001
2016-05-18 01:21:16,055 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D1DAA64622724BED8C1F2A2E7A2F1728 of vertex/vertexGroup:BAF3578B26124890A87F4BBBC7BD4EF2 isVertexGroupOutput:false
2016-05-18 01:21:16,055 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:16,056 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534475789_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:16,057 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534475789_0001_00_000001 completed successfully
2016-05-18 01:21:16,057 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534475789_0001_1, queueSize=0
2016-05-18 01:21:16,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534475789_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534475789_0001_1, startTime=1463534475969, finishTime=1463534476055, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=150, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476030, Process_Duration=4, Process_End_Time=1463534476034, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:21:16,058 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534475789_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:16,058 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534475789_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:16 Completed Dag: dag_1463534475789_0001_1
2016-05-18 01:21:16,059 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534475789_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:16,059 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534475789_0001_1_post
2016-05-18 01:21:16,059 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,059 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:16,059 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:16,059 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap, with id=dag_1463534475789_0001_1
2016-05-18 01:21:16,059 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF] test/(1/1) ...pesplatform/testSwap/swap, with id=dag_1463534475789_0001_1
2016-05-18 01:21:16,066 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1463534475789_0001
2016-05-18 01:21:16,070 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:16,070 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:16,070 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:16,070 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:16,070 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:16,071 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:16,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:16,071 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:16,072 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:16,072 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:16,072 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:16,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:16,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:16,076 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:16,080 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/20FF32662B914EEBB83B0E635BABB136/C143B4DF8241437786227748CDEC32EF/.tez/application_1463534475789_0001
2016-05-18 01:21:16,080 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap/_temporary
2016-05-18 01:21:16,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.315
2016-05-18 01:21:16,127 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-05-18 01:21:16,162 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:16,162 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:16,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 01:21:16,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 01:21:16,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:16,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:16,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:16,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt
2016-05-18 01:21:16,249 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-18 01:21:16,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:16,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:21:16,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-18 01:21:16,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:16,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:16,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:16,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-05-18 01:21:16,271 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:16,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:16,299 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:16,300 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:16,300 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:16,309 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001 doesn't exist and is created
2016-05-18 01:21:16,335 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:16,335 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:16,336 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001_wd
2016-05-18 01:21:16,354 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534476300_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:16,354 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:16,354 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:16,356 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:16,357 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:16,357 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:16,357 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:16,358 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:16,360 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:16,361 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:16,361 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:16,361 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:16,361 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:16,362 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:16,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:16,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:16,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:16,363 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:16,363 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:16,363 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:16,366 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:16,367 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:16,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:16,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001_wd/localmode-log-dir
2016-05-18 01:21:16,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001_wd/localmode-log-dir/history.txt.appattempt_1463534476300_0001_000000, maxErrors=10
2016-05-18 01:21:16,369 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:16,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534476300_0001_000000, appSubmitTime=1463534476353, launchTime=1463534476353
2016-05-18 01:21:16,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:16,382 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45581
2016-05-18 01:21:16,383 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534476300
2016-05-18 01:21:16,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534476300_0001_000000, startTime=1463534476386
2016-05-18 01:21:16,386 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:16,436 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:16,436 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:16,436 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463534476300_0001, dagName=[E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-18 01:21:16,464 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-18 01:21:16,464 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:16,465 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534476300_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001_wd/localmode-log-dir/dag_1463534476300_0001_1.dot
2016-05-18 01:21:16,466 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001_wd/localmode-log-dir/dag_1463534476300_0001_1-tez-dag.pb.txt
2016-05-18 01:21:16,471 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534476300_0001_1
2016-05-18 01:21:16,472 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,472 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-18 01:21:16 Running Dag: dag_1463534476300_0001_1
2016-05-18 01:21:16,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534476300_0001_1, submitTime=1463534476464
2016-05-18 01:21:16,489 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001/tez-conf.pb" } size: 40949 timestamp: 1463534476000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:16,489 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,489 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,490 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:16,490 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:16,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534476300_0001_1, initTime=1463534476490
2016-05-18 01:21:16,491 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534476300_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:16,491 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:16,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:DAG_STARTED]: dagID=dag_1463534476300_0001_1, startTime=1463534476492
2016-05-18 01:21:16,492 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534476300_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:16,492 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F3BC3D1192CF46D19079BD00DDEA7005 : {F2AEFDE2EAB04D6FBD630A7BC609B720={InputName=F2AEFDE2EAB04D6FBD630A7BC609B720}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:16,492 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: F2AEFDE2EAB04D6FBD630A7BC609B720, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:16,492 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,492 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,492 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,493 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,493 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:16,493 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463534476300_0001, dagName=[E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped
2016-05-18 01:21:16,494 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: F2AEFDE2EAB04D6FBD630A7BC609B720 on vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,497 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input F2AEFDE2EAB04D6FBD630A7BC609B720 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:16,500 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:16,500 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 30 Min splitLength: 52428800 New desired splits: 1 Total length: 30 Original splits: 2
2016-05-18 01:21:16,500 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 30 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 30 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:16,500 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:16,501 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: F2AEFDE2EAB04D6FBD630A7BC609B720 on vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,502 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:16,502 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {F2AEFDE2EAB04D6FBD630A7BC609B720=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:16,503 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] parallelism set to 1
2016-05-18 01:21:16,503 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:16,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005], numAdditionalOutputs=1
2016-05-18 01:21:16,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=7C573A3D604546A198DF2E2F66F062BC, vertex=vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:16,507 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F3BC3D1192CF46D19079BD00DDEA7005:7C573A3D604546A198DF2E2F66F062BC using oldmapred API
2016-05-18 01:21:16,507 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=7C573A3D604546A198DF2E2F66F062BC, vertexName=F3BC3D1192CF46D19079BD00DDEA7005, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:16,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, vertexId=vertex_1463534476300_0001_1_00, initRequestedTime=1463534476492, initedTime=1463534476504, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:16,509 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] with distanceFromRoot: 0
2016-05-18 01:21:16,509 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:16,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534476300_0001_1_00, startRequestedTime=1463534476493, startedTime=1463534476510
2016-05-18 01:21:16,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534476300_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:16,511 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:16,512 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F3BC3D1192CF46D19079BD00DDEA7005
2016-05-18 01:21:16,512 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:TASK_STARTED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, taskId=task_1463534476300_0001_1_00_000000, scheduledTime=1463534476512, launchTime=1463534476512
2016-05-18 01:21:16,513 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,514 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:16,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534476300_0001_00_000001, launchTime=1463534476516
2016-05-18 01:21:16,517 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534476300_0001_00_000001
2016-05-18 01:21:16,518 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534476300_0001_1_00_000000_0] started. Is using containerId: [container_1463534476300_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:16,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0, startTime=1463534476518, containerId=container_1463534476300_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:16,519 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534476300_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16 Starting to run new task attempt: attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,521 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,521 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,521 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:16,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: F3BC3D1192CF46D19079BD00DDEA7005, VertexParallelism: 1, TaskAttemptID:attempt_1463534476300_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F2AEFDE2EAB04D6FBD630A7BC609B720, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=7C573A3D604546A198DF2E2F66F062BC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:16,523 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:16,524 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,526 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534476300_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:16,526 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534476300_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:16,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534476300_0001, with dag id: dag_1463534476300_0001_1
2016-05-18 01:21:16,541 INFO input.MRInput (MRInput.java:initialize(440)) - F2AEFDE2EAB04D6FBD630A7BC609B720 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:16,542 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: F2AEFDE2EAB04D6FBD630A7BC609B720
2016-05-18 01:21:16,542 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:16,552 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F3BC3D1192CF46D19079BD00DDEA7005, ordinal: 0
2016-05-18 01:21:16,553 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F3BC3D1192CF46D19079BD00DDEA7005, mem on start (mb), free: 92, total: 589, max: 672
2016-05-18 01:21:16,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:16,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:16,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:16,557 INFO output.MROutput (MROutput.java:initialize(421)) - 7C573A3D604546A198DF2E2F66F062BC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:16,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:16,557 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F2AEFDE2EAB04D6FBD630A7BC609B720:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [7C573A3D604546A198DF2E2F66F062BC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:16,558 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:16,558 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:16,558 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:21:16,558 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F2AEFDE2EAB04D6FBD630A7BC609B720:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [7C573A3D604546A198DF2E2F66F062BC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:16,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:16,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F2AEFDE2EAB04D6FBD630A7BC609B720 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:16,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:16,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:16,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F2AEFDE2EAB04D6FBD630A7BC609B720
2016-05-18 01:21:16,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:16,560 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,564 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:0+15, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt:15+15], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=30}
2016-05-18 01:21:16,565 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - F2AEFDE2EAB04D6FBD630A7BC609B720 initialized RecordReader from event
2016-05-18 01:21:16,568 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:21:16,569 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: F2AEFDE2EAB04D6FBD630A7BC609B720
2016-05-18 01:21:16,569 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: 7C573A3D604546A198DF2E2F66F062BC
2016-05-18 01:21:16,569 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-05-18 01:21:16,570 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower+upper.txt"]
2016-05-18 01:21:16,570 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F3BC3D1192CF46D19079BD00DDEA7005, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:16,573 INFO output.MROutput (MROutput.java:close(526)) - 7C573A3D604546A198DF2E2F66F062BC closed
2016-05-18 01:21:16,573 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534476300_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:21:16,574 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,574 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F3BC3D1192CF46D19079BD00DDEA7005, mem on close (mb), free: 90, total: 589, max: 672
2016-05-18 01:21:16,575 INFO output.MROutput (MROutput.java:close(526)) - 7C573A3D604546A198DF2E2F66F062BC closed
2016-05-18 01:21:16,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:16 Completed running task attempt: attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,576 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:16,576 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534476300_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:16,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534476300_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463534476570, Process_Duration=4, Process_End_Time=1463534476574, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-05-18 01:21:16,578 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534476300_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:16,578 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:16,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:16,579 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, taskAttemptId=attempt_1463534476300_0001_1_00_000000_0, creationTime=1463534476513, allocationTime=1463534476515, startTime=1463534476518, finishTime=1463534476578, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:16,579 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534476300_0001_00_000001
2016-05-18 01:21:16,579 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534476300_0001_1_00_000000_0
2016-05-18 01:21:16,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:TASK_FINISHED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, taskId=task_1463534476300_0001_1_00_000000, startTime=1463534476518, finishTime=1463534476579, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463534476300_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476570, Process_Duration=4, Process_End_Time=1463534476574, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:21:16,580 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:16,580 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005]
2016-05-18 01:21:16,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:VERTEX_FINISHED]: vertexName=F3BC3D1192CF46D19079BD00DDEA7005, vertexId=vertex_1463534476300_0001_1_00, initRequestedTime=1463534476492, initedTime=1463534476504, startRequestedTime=1463534476493, startedTime=1463534476510, finishTime=1463534476580, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476570, Process_Duration=4, Process_End_Time=1463534476574, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1463534476518, firstTasksToStart=[ task_1463534476300_0001_1_00_000000 ], lastTaskFinishTime=1463534476578, lastTasksToFinish=[ task_1463534476300_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534476300_0001_1_00_000000 ], longestDurationTasks=[ task_1463534476300_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:16,582 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:16,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534476300_0001_00_000001, stoppedTime=1463534476582, exitStatus=0
2016-05-18 01:21:16,582 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:21:16,583 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:16,583 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534476300_0001_1
2016-05-18 01:21:16,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534476300_0001_1
2016-05-18 01:21:16,583 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 7C573A3D604546A198DF2E2F66F062BC for vertex: vertex_1463534476300_0001_1_00 [F3BC3D1192CF46D19079BD00DDEA7005], outputName: 7C573A3D604546A198DF2E2F66F062BC
2016-05-18 01:21:16,583 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534476300_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:16,584 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534476300_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:16,591 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534476300_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:16,591 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534476300_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:16,591 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534476300_0001_00_000001, Exiting
2016-05-18 01:21:16,591 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534476300_0001_00_000001
2016-05-18 01:21:16,591 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534476300_0001_00_000001
2016-05-18 01:21:16,592 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534476300_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:16,592 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534476300_0001_00_000001 completed successfully
2016-05-18 01:21:16,594 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:7C573A3D604546A198DF2E2F66F062BC of vertex/vertexGroup:F3BC3D1192CF46D19079BD00DDEA7005 isVertexGroupOutput:false
2016-05-18 01:21:16,595 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:16,597 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534476300_0001_1, queueSize=0
2016-05-18 01:21:16,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534476300_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534476300_0001_1, startTime=1463534476491, finishTime=1463534476595, timeTaken=104, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534476570, Process_Duration=4, Process_End_Time=1463534476574, Read_Duration=1, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-05-18 01:21:16,599 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534476300_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:16,599 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534476300_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:16 Completed Dag: dag_1463534476300_0001_1
2016-05-18 01:21:16,600 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534476300_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:16,601 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534476300_0001_1_post
2016-05-18 01:21:16,601 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:16,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:16,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:16,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped, with id=dag_1463534476300_0001_1
2016-05-18 01:21:16,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1] test/(1/1) ...orm/testUnGroup/ungrouped, with id=dag_1463534476300_0001_1
2016-05-18 01:21:16,605 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1463534476300_0001
2016-05-18 01:21:16,608 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:16,608 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:16,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:16,609 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:16,609 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:16,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:16,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:16,611 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:16,611 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:16,611 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:16,611 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:16,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:16,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:16,613 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:16,617 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/E891ADFC57334623BBB3276FFF72D37A/CD81A5A4B74D4DA8AB49FFE5911248D1/.tez/application_1463534476300_0001
2016-05-18 01:21:16,618 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped/_temporary
2016-05-18 01:21:16,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.349
2016-05-18 01:21:16,674 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-05-18 01:21:16,722 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:16,722 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:16,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-18 01:21:16,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.047
2016-05-18 01:21:16,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:16,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:16,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:16,844 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt
2016-05-18 01:21:16,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:21:16,903 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-18 01:21:17,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-18 01:21:17,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:17,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:17,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-18 01:21:17,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-18 01:21:17,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-18 01:21:17,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-18 01:21:17,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-05-18 01:21:17,019 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:17,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-18 01:21:17,042 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:17,043 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:17,043 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:17,052 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001 doesn't exist and is created
2016-05-18 01:21:17,075 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:17,075 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:17,076 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd
2016-05-18 01:21:17,092 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534477042_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:17,092 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:17,092 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:17,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:17,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:17,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:17,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:17,095 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:17,096 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:17,096 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:17,096 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:17,096 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:17,096 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:17,097 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:17,097 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:17,097 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:17,097 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:17,097 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:17,097 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:17,097 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:17,099 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:17,100 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:17,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:17,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-log-dir
2016-05-18 01:21:17,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-log-dir/history.txt.appattempt_1463534477042_0001_000000, maxErrors=10
2016-05-18 01:21:17,101 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:17,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534477042_0001_000000, appSubmitTime=1463534477091, launchTime=1463534477091
2016-05-18 01:21:17,107 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41518
2016-05-18 01:21:17,108 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534477042
2016-05-18 01:21:17,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:17,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534477042_0001_000000, startTime=1463534477111
2016-05-18 01:21:17,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:17,175 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:17,176 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:17,176 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463534477042_0001, dagName=[1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-18 01:21:17,403 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-18 01:21:17,403 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:17,404 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534477042_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-log-dir/dag_1463534477042_0001_1.dot
2016-05-18 01:21:17,406 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-log-dir/dag_1463534477042_0001_1-tez-dag.pb.txt
2016-05-18 01:21:17,418 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534477042_0001_1
2016-05-18 01:21:17,418 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:17,418 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-18 01:21:17 Running Dag: dag_1463534477042_0001_1
2016-05-18 01:21:17,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534477042_0001_1, submitTime=1463534477403
2016-05-18 01:21:17,437 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001/tez-conf.pb" } size: 40949 timestamp: 1463534477000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:17,437 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,438 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:17,439 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001/tez-conf.pb" } size: 40949 timestamp: 1463534477000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:17,439 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,439 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:17,440 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001/tez-conf.pb" } size: 40949 timestamp: 1463534477000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:17,440 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:17,441 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:17,441 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:17,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534477042_0001_1, initTime=1463534477442
2016-05-18 01:21:17,442 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534477042_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:17,443 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:17,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:DAG_STARTED]: dagID=dag_1463534477042_0001_1, startTime=1463534477443
2016-05-18 01:21:17,444 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534477042_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:17,444 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F718CBD99C794830A157279577BBF390 : {D2C275E51045448A9689E454287C06D9={InputName=D2C275E51045448A9689E454287C06D9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:17,444 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: D2C275E51045448A9689E454287C06D9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:17,444 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,444 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,444 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,445 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,445 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:17,445 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A7414596E620413586EA78F36907996C : {EB91569B7D0848B680B28B1BA9DBB393={InputName=EB91569B7D0848B680B28B1BA9DBB393}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:17,445 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: EB91569B7D0848B680B28B1BA9DBB393, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:17,446 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,446 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,446 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,446 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,446 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:17,447 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:17,448 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463534477042_0001, dagName=[1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-05-18 01:21:17,450 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: D2C275E51045448A9689E454287C06D9 on vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,451 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:17,451 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:17,452 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:17,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534477042_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:17,456 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563], numAdditionalOutputs=1
2016-05-18 01:21:17,457 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C2AF9CA230EA4A8A8A3D486482D42354, vertex=vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:17,458 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: EB91569B7D0848B680B28B1BA9DBB393 on vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,464 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input D2C275E51045448A9689E454287C06D9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:17,465 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input EB91569B7D0848B680B28B1BA9DBB393 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:17,468 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:17,468 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:17,463 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B7EC8D8BAC9C4806B5FC9426B6832563:C2AF9CA230EA4A8A8A3D486482D42354 using oldmapred API
2016-05-18 01:21:17,469 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C2AF9CA230EA4A8A8A3D486482D42354, vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:17,469 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:17,470 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:17,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, vertexId=vertex_1463534477042_0001_1_02, initRequestedTime=1463534477447, initedTime=1463534477456, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:17,470 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:17,471 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:17,472 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:17,472 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: EB91569B7D0848B680B28B1BA9DBB393 on vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,473 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:17,474 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:17,474 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: D2C275E51045448A9689E454287C06D9 on vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,477 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:17,478 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463534477042_0001, with dag id: dag_1463534477042_0001_1
2016-05-18 01:21:17,477 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:17,478 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {EB91569B7D0848B680B28B1BA9DBB393=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:17,481 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] parallelism set to 1
2016-05-18 01:21:17,482 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:17,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A7414596E620413586EA78F36907996C, vertexId=vertex_1463534477042_0001_1_01, initRequestedTime=1463534477445, initedTime=1463534477483, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:17,483 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] with distanceFromRoot: 0
2016-05-18 01:21:17,483 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:17,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534477042_0001_1_01, startRequestedTime=1463534477446, startedTime=1463534477483
2016-05-18 01:21:17,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534477042_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:17,484 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:17,484 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534477042_0001_1_01 for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] numStartedSources: 1 numSources: 2
2016-05-18 01:21:17,484 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] numStartedSources: 1 numSources: 2
2016-05-18 01:21:17,481 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {D2C275E51045448A9689E454287C06D9=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:17,485 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A7414596E620413586EA78F36907996C
2016-05-18 01:21:17,485 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] parallelism set to 1
2016-05-18 01:21:17,485 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:17,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F718CBD99C794830A157279577BBF390, vertexId=vertex_1463534477042_0001_1_00, initRequestedTime=1463534477444, initedTime=1463534477485, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:17,485 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,486 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] with distanceFromRoot: 0
2016-05-18 01:21:17,486 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:17,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534477042_0001_1_00, startRequestedTime=1463534477446, startedTime=1463534477486
2016-05-18 01:21:17,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534477042_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:17,487 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:17,487 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534477042_0001_1_00 for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] numStartedSources: 2 numSources: 2
2016-05-18 01:21:17,487 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] with distanceFromRoot: 1
2016-05-18 01:21:17,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534477042_0001_1_02, startRequestedTime=1463534477487, startedTime=1463534477487
2016-05-18 01:21:17,488 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F718CBD99C794830A157279577BBF390
2016-05-18 01:21:17,493 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,488 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:17,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_STARTED]: vertexName=F718CBD99C794830A157279577BBF390, taskId=task_1463534477042_0001_1_00_000000, scheduledTime=1463534477494, launchTime=1463534477494
2016-05-18 01:21:17,495 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:17,498 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:17,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_STARTED]: vertexName=A7414596E620413586EA78F36907996C, taskId=task_1463534477042_0001_1_01_000000, scheduledTime=1463534477498, launchTime=1463534477498
2016-05-18 01:21:17,499 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: B7EC8D8BAC9C4806B5FC9426B6832563 with 0 source tasks and 1 pending tasks
2016-05-18 01:21:17,500 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:17,501 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: F718CBD99C794830A157279577BBF390 in vertex: B7EC8D8BAC9C4806B5FC9426B6832563 numBipartiteSourceTasks: 1
2016-05-18 01:21:17,501 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A7414596E620413586EA78F36907996C in vertex: B7EC8D8BAC9C4806B5FC9426B6832563 numBipartiteSourceTasks: 2
2016-05-18 01:21:17,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534477042_0001_00_000001, launchTime=1463534477502
2016-05-18 01:21:17,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534477042_0001_00_000002, launchTime=1463534477504
2016-05-18 01:21:17,506 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000001
2016-05-18 01:21:17,506 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000002
2016-05-18 01:21:17,509 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534477042_0001_1_00_000000_0] started. Is using containerId: [container_1463534477042_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:17,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F718CBD99C794830A157279577BBF390, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0, startTime=1463534477508, containerId=container_1463534477042_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:17,509 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:17,509 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534477042_0001_1_01_000000_0] started. Is using containerId: [container_1463534477042_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:17,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A7414596E620413586EA78F36907996C, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0, startTime=1463534477509, containerId=container_1463534477042_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:17 Starting to run new task attempt: attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:17,510 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17 Starting to run new task attempt: attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17,510 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17,510 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:17,510 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:17,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: A7414596E620413586EA78F36907996C, VertexParallelism: 1, TaskAttemptID:attempt_1463534477042_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EB91569B7D0848B680B28B1BA9DBB393, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B7EC8D8BAC9C4806B5FC9426B6832563, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:17,510 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:17,511 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:17,511 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:17,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: F718CBD99C794830A157279577BBF390, VertexParallelism: 1, TaskAttemptID:attempt_1463534477042_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2C275E51045448A9689E454287C06D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B7EC8D8BAC9C4806B5FC9426B6832563, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:17,511 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:17,511 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:17,511 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:17,514 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534477042_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:17,514 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534477042_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:17,515 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17,517 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534477042_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:21:17,518 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534477042_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:17,541 INFO input.MRInput (MRInput.java:initialize(440)) - D2C275E51045448A9689E454287C06D9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:17,541 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: D2C275E51045448A9689E454287C06D9
2016-05-18 01:21:17,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:17,544 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:17,546 INFO input.MRInput (MRInput.java:initialize(440)) - EB91569B7D0848B680B28B1BA9DBB393 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:17,548 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: EB91569B7D0848B680B28B1BA9DBB393
2016-05-18 01:21:17,554 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F718CBD99C794830A157279577BBF390, ordinal: 0
2016-05-18 01:21:17,554 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F718CBD99C794830A157279577BBF390, mem on start (mb), free: 176, total: 590, max: 672
2016-05-18 01:21:17,554 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A7414596E620413586EA78F36907996C, ordinal: 1
2016-05-18 01:21:17,554 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A7414596E620413586EA78F36907996C, mem on start (mb), free: 176, total: 590, max: 672
2016-05-18 01:21:17,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:17,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:17,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:17,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:17,555 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[EB91569B7D0848B680B28B1BA9DBB393:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [B7EC8D8BAC9C4806B5FC9426B6832563:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:17,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:17,556 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:17,557 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:17,557 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:17,557 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[EB91569B7D0848B680B28B1BA9DBB393:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [B7EC8D8BAC9C4806B5FC9426B6832563:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:17,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:17,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: EB91569B7D0848B680B28B1BA9DBB393 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:17,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:17,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:17,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: EB91569B7D0848B680B28B1BA9DBB393
2016-05-18 01:21:17,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:17,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:17,559 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17,562 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:17,563 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:17,563 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - EB91569B7D0848B680B28B1BA9DBB393 initialized RecordReader from event
2016-05-18 01:21:17,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:17,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:17,565 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[D2C275E51045448A9689E454287C06D9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [B7EC8D8BAC9C4806B5FC9426B6832563:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:17,565 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: EB91569B7D0848B680B28B1BA9DBB393
2016-05-18 01:21:17,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:17,566 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 81DCE0BA00524F8C82D692EA3B230B33
2016-05-18 01:21:17,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:17,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:17,566 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[D2C275E51045448A9689E454287C06D9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [B7EC8D8BAC9C4806B5FC9426B6832563:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:17,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:17,566 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 81DCE0BA00524F8C82D692EA3B230B33
2016-05-18 01:21:17,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D2C275E51045448A9689E454287C06D9
2016-05-18 01:21:17,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D2C275E51045448A9689E454287C06D9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:17,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:17,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:17,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:17,569 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:17,570 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - B7EC8D8BAC9C4806B5FC9426B6832563 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@1f5ce838, comparator=cascading.tuple.hadoop.util.TupleComparator@36036784, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:17,570 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for B7EC8D8BAC9C4806B5FC9426B6832563: , UsingHashComparator=false
2016-05-18 01:21:17,590 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:17,591 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:17,591 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for B7EC8D8BAC9C4806B5FC9426B6832563: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:17,591 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - B7EC8D8BAC9C4806B5FC9426B6832563: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:17,591 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:17,591 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:17,592 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"] streamed: true, id: D2C275E51045448A9689E454287C06D9
2016-05-18 01:21:17,592 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: 81DCE0BA00524F8C82D692EA3B230B33
2016-05-18 01:21:17,592 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 81DCE0BA00524F8C82D692EA3B230B33
2016-05-18 01:21:17,592 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A7414596E620413586EA78F36907996C, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:17,598 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:17,598 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - D2C275E51045448A9689E454287C06D9 initialized RecordReader from event
2016-05-18 01:21:17,599 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:17,599 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A7414596E620413586EA78F36907996C, mem on close (mb), free: 72, total: 590, max: 672
2016-05-18 01:21:17,598 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - B7EC8D8BAC9C4806B5FC9426B6832563 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4076dc80, comparator=cascading.tuple.hadoop.util.TupleComparator@511c8edc, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:17,599 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for B7EC8D8BAC9C4806B5FC9426B6832563: , UsingHashComparator=false
2016-05-18 01:21:18,038 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - B7EC8D8BAC9C4806B5FC9426B6832563: Starting flush of map output
2016-05-18 01:21:18,039 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - B7EC8D8BAC9C4806B5FC9426B6832563: Span0.length = 5, perItem = 15
2016-05-18 01:21:18,039 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - B7EC8D8BAC9C4806B5FC9426B6832563: done sorting span=0, length=5, time=0
2016-05-18 01:21:18,041 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - B7EC8D8BAC9C4806B5FC9426B6832563: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:21:18,043 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - B7EC8D8BAC9C4806B5FC9426B6832563: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-local-dir/output/attempt_1463534477042_0001_1_01_000000_0_10026_0/file.out
2016-05-18 01:21:18,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:18,052 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:18,053 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:18,053 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for B7EC8D8BAC9C4806B5FC9426B6832563: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:18,053 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - B7EC8D8BAC9C4806B5FC9426B6832563: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:18 Completed running task attempt: attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:18,054 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:18,055 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:18,055 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534477042_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:18,055 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F718CBD99C794830A157279577BBF390, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:21:18,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534477042_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534477593, Process_Duration=6, Process_End_Time=1463534477599, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:18,059 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:18,060 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534477042_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,060 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A7414596E620413586EA78F36907996C, taskAttemptId=attempt_1463534477042_0001_1_01_000000_0, creationTime=1463534477499, allocationTime=1463534477503, startTime=1463534477509, finishTime=1463534478060, timeTaken=551, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:18,060 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:18,061 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,061 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534477042_0001_1_01_000000_0
2016-05-18 01:21:18,061 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F718CBD99C794830A157279577BBF390, mem on close (mb), free: 227, total: 450, max: 672
2016-05-18 01:21:18,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_FINISHED]: vertexName=A7414596E620413586EA78F36907996C, taskId=task_1463534477042_0001_1_01_000000, startTime=1463534477509, finishTime=1463534478061, timeTaken=552, status=SUCCEEDED, successfulAttemptID=attempt_1463534477042_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534477593, Process_Duration=6, Process_End_Time=1463534477599, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:18,063 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:18,063 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:18,063 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000002
2016-05-18 01:21:18,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_FINISHED]: vertexName=A7414596E620413586EA78F36907996C, vertexId=vertex_1463534477042_0001_1_01, initRequestedTime=1463534477445, initedTime=1463534477483, startRequestedTime=1463534477446, startedTime=1463534477483, finishTime=1463534478063, timeTaken=580, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534477593, Process_Duration=6, Process_End_Time=1463534477599, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534477509, firstTasksToStart=[ task_1463534477042_0001_1_01_000000 ], lastTaskFinishTime=1463534478060, lastTasksToFinish=[ task_1463534477042_0001_1_01_000000 ], minTaskDuration=551, maxTaskDuration=551, avgTaskDuration=551.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534477042_0001_1_01_000000 ], longestDurationTasks=[ task_1463534477042_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:18,064 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:18,065 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] attempt: attempt_1463534477042_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:18,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534477042_0001_00_000002, stoppedTime=1463534478065, exitStatus=0
2016-05-18 01:21:18,066 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:18,066 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:18,062 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - B7EC8D8BAC9C4806B5FC9426B6832563: Starting flush of map output
2016-05-18 01:21:18,066 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534477042_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:18,066 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - B7EC8D8BAC9C4806B5FC9426B6832563: Span0.length = 5, perItem = 15
2016-05-18 01:21:18,067 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - B7EC8D8BAC9C4806B5FC9426B6832563: done sorting span=0, length=5, time=1
2016-05-18 01:21:18,067 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - B7EC8D8BAC9C4806B5FC9426B6832563: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:21:18,068 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - B7EC8D8BAC9C4806B5FC9426B6832563: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-local-dir/output/attempt_1463534477042_0001_1_00_000000_0_10024_0/file.out
2016-05-18 01:21:18,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:18 Completed running task attempt: attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:18,072 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:18,072 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534477042_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:18,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534477042_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534478055, Process_Duration=4, Process_End_Time=1463534478059, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:18,073 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534477042_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,073 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:18,074 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F718CBD99C794830A157279577BBF390, taskAttemptId=attempt_1463534477042_0001_1_00_000000_0, creationTime=1463534477494, allocationTime=1463534477502, startTime=1463534477508, finishTime=1463534478073, timeTaken=565, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:18,074 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000001
2016-05-18 01:21:18,074 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534477042_0001_1_00_000000_0
2016-05-18 01:21:18,074 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534477042_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:18,074 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:18,075 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534477042_0001_00_000002, Exiting
2016-05-18 01:21:18,075 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534477042_0001_00_000002
2016-05-18 01:21:18,075 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534477042_0001_00_000002
2016-05-18 01:21:18,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_FINISHED]: vertexName=F718CBD99C794830A157279577BBF390, taskId=task_1463534477042_0001_1_00_000000, startTime=1463534477508, finishTime=1463534478074, timeTaken=566, status=SUCCEEDED, successfulAttemptID=attempt_1463534477042_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534478055, Process_Duration=4, Process_End_Time=1463534478059, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:18,075 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:18,076 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:18,076 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534477042_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:18,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534477042_0001_00_000002 completed successfully
2016-05-18 01:21:18,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_FINISHED]: vertexName=F718CBD99C794830A157279577BBF390, vertexId=vertex_1463534477042_0001_1_00, initRequestedTime=1463534477444, initedTime=1463534477485, startRequestedTime=1463534477446, startedTime=1463534477486, finishTime=1463534478076, timeTaken=590, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534478055, Process_Duration=4, Process_End_Time=1463534478059, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534477508, firstTasksToStart=[ task_1463534477042_0001_1_00_000000 ], lastTaskFinishTime=1463534478073, lastTasksToFinish=[ task_1463534477042_0001_1_00_000000 ], minTaskDuration=565, maxTaskDuration=565, avgTaskDuration=565.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534477042_0001_1_00_000000 ], longestDurationTasks=[ task_1463534477042_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:18,078 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:18,078 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] attempt: attempt_1463534477042_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:18,078 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B7EC8D8BAC9C4806B5FC9426B6832563
2016-05-18 01:21:18,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534477042_0001_00_000001, stoppedTime=1463534478078, exitStatus=0
2016-05-18 01:21:18,079 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A7414596E620413586EA78F36907996C, destinationVertexName=B7EC8D8BAC9C4806B5FC9426B6832563. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:18,079 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=F718CBD99C794830A157279577BBF390, destinationVertexName=B7EC8D8BAC9C4806B5FC9426B6832563. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:18,079 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:18,080 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:18,080 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:18,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_STARTED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, taskId=task_1463534477042_0001_1_02_000000, scheduledTime=1463534478081, launchTime=1463534478081
2016-05-18 01:21:18,082 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:18,082 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534477042_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:18,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534477042_0001_00_000003, launchTime=1463534478084
2016-05-18 01:21:18,084 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000003
2016-05-18 01:21:18,085 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534477042_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:18,085 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:18,085 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534477042_0001_00_000001, Exiting
2016-05-18 01:21:18,085 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534477042_0001_00_000001
2016-05-18 01:21:18,085 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534477042_0001_00_000001
2016-05-18 01:21:18,085 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534477042_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:18,086 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534477042_0001_00_000001 completed successfully
2016-05-18 01:21:18,086 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534477042_0001_1_02_000000_0] started. Is using containerId: [container_1463534477042_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:21:18,087 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0, startTime=1463534478086, containerId=container_1463534477042_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:21:18,087 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18 Starting to run new task attempt: attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,088 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,089 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,089 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:18,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: B7EC8D8BAC9C4806B5FC9426B6832563, VertexParallelism: 1, TaskAttemptID:attempt_1463534477042_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A7414596E620413586EA78F36907996C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=F718CBD99C794830A157279577BBF390, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C2AF9CA230EA4A8A8A3D486482D42354, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:18,090 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:18,091 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,093 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:18,095 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534477042_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:18,095 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:18,095 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534477042_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:21:18,096 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:18,105 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B7EC8D8BAC9C4806B5FC9426B6832563, ordinal: 2
2016-05-18 01:21:18,105 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7EC8D8BAC9C4806B5FC9426B6832563, mem on start (mb), free: 220, total: 450, max: 672
2016-05-18 01:21:18,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:18,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:21:18,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:18,106 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:18,112 INFO output.MROutput (MROutput.java:initialize(421)) - C2AF9CA230EA4A8A8A3D486482D42354: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:18,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:18,113 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F718CBD99C794830A157279577BBF390:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [A7414596E620413586EA78F36907996C:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [C2AF9CA230EA4A8A8A3D486482D42354:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:18,113 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:18,113 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:21:18,113 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:21:18,113 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F718CBD99C794830A157279577BBF390:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [A7414596E620413586EA78F36907996C:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [C2AF9CA230EA4A8A8A3D486482D42354:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:18,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:18,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A7414596E620413586EA78F36907996C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:18,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F718CBD99C794830A157279577BBF390 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:18,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:21:18,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:21:18,114 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A7414596E620413586EA78F36907996C: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:18,115 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A7414596E620413586EA78F36907996C: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:18,116 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - F718CBD99C794830A157279577BBF390: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:18,116 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - F718CBD99C794830A157279577BBF390: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:18,116 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A7414596E620413586EA78F36907996C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:18,117 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: F718CBD99C794830A157279577BBF390 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:18,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F718CBD99C794830A157279577BBF390
2016-05-18 01:21:18,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A7414596E620413586EA78F36907996C
2016-05-18 01:21:18,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:18,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:18,122 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,123 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A7414596E620413586EA78F36907996C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:18,124 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F718CBD99C794830A157279577BBF390: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:18,125 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534477042_0001_1_01_000000_0_10026} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463534478125, TimeTaken=1, Rate=0.09 MB/s
2016-05-18 01:21:18,125 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A7414596E620413586EA78F36907996C
2016-05-18 01:21:18,125 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:18,125 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A7414596E620413586EA78F36907996C, wasInterrupted=false
2016-05-18 01:21:18,127 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:18,127 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534477042_0001_1_00_000000_0_10024} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463534478127, TimeTaken=3, Rate=0.03 MB/s
2016-05-18 01:21:18,127 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : F718CBD99C794830A157279577BBF390
2016-05-18 01:21:18,128 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-05-18 01:21:18,128 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:18,127 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:18,128 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: F718CBD99C794830A157279577BBF390, wasInterrupted=false
2016-05-18 01:21:18,128 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: 81DCE0BA00524F8C82D692EA3B230B33
2016-05-18 01:21:18,129 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:18,129 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: C2AF9CA230EA4A8A8A3D486482D42354
2016-05-18 01:21:18,129 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-05-18 01:21:18,130 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 81DCE0BA00524F8C82D692EA3B230B33, for 2 inputs
2016-05-18 01:21:18,130 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:18,130 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-local-dir/output/attempt_1463534477042_0001_1_01_000000_0_10026/file.out, len=95, isLocal=true
2016-05-18 01:21:18,132 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:21:18,131 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] 81DCE0BA00524F8C82D692EA3B230B33, for 2 inputs
2016-05-18 01:21:18,132 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:18,132 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001_wd/localmode-local-dir/output/attempt_1463534477042_0001_1_00_000000_0_10024/file.out, len=95, isLocal=true
2016-05-18 01:21:18,132 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:21:18,133 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:18,133 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:18,132 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:18,133 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:21:18,133 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : F718CBD99C794830A157279577BBF390
2016-05-18 01:21:18,134 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - F718CBD99C794830A157279577BBF390: Shuffle Runner thread complete
2016-05-18 01:21:18,134 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:21:18,136 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A7414596E620413586EA78F36907996C
2016-05-18 01:21:18,136 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A7414596E620413586EA78F36907996C: Shuffle Runner thread complete
2016-05-18 01:21:18,139 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:21:18,139 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:21:18,143 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B7EC8D8BAC9C4806B5FC9426B6832563, all 2 inputs ready in: 00:00:00.000
2016-05-18 01:21:18,144 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A7414596E620413586EA78F36907996C: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:18,145 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - F718CBD99C794830A157279577BBF390: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:18,152 INFO output.MROutput (MROutput.java:close(526)) - C2AF9CA230EA4A8A8A3D486482D42354 closed
2016-05-18 01:21:18,153 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534477042_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:21:18,154 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,154 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7EC8D8BAC9C4806B5FC9426B6832563, mem on close (mb), free: 214, total: 450, max: 672
2016-05-18 01:21:18,154 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A7414596E620413586EA78F36907996C
2016-05-18 01:21:18,154 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A7414596E620413586EA78F36907996C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:18,154 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: F718CBD99C794830A157279577BBF390
2016-05-18 01:21:18,154 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F718CBD99C794830A157279577BBF390: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:18,154 INFO output.MROutput (MROutput.java:close(526)) - C2AF9CA230EA4A8A8A3D486482D42354 closed
2016-05-18 01:21:18,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:18 Completed running task attempt: attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,155 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:21:18,155 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534477042_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:21:18,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534477042_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534478143, Process_Duration=11, Process_End_Time=1463534478154, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:21:18,156 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534477042_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, taskAttemptId=attempt_1463534477042_0001_1_02_000000_0, creationTime=1463534478081, allocationTime=1463534478083, startTime=1463534478086, finishTime=1463534478156, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:18,157 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:18,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:18,157 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534477042_0001_1_02_000000_0
2016-05-18 01:21:18,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:TASK_FINISHED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, taskId=task_1463534477042_0001_1_02_000000, startTime=1463534478086, finishTime=1463534478157, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463534477042_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534478143, Process_Duration=11, Process_End_Time=1463534478154, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:21:18,158 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:18,158 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563]
2016-05-18 01:21:18,157 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,159 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534477042_0001_00_000003
2016-05-18 01:21:18,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:VERTEX_FINISHED]: vertexName=B7EC8D8BAC9C4806B5FC9426B6832563, vertexId=vertex_1463534477042_0001_1_02, initRequestedTime=1463534477447, initedTime=1463534477456, startRequestedTime=1463534477487, startedTime=1463534477487, finishTime=1463534478158, timeTaken=671, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534478143, Process_Duration=11, Process_End_Time=1463534478154, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463534478086, firstTasksToStart=[ task_1463534477042_0001_1_02_000000 ], lastTaskFinishTime=1463534478156, lastTasksToFinish=[ task_1463534477042_0001_1_02_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534477042_0001_1_02_000000 ], longestDurationTasks=[ task_1463534477042_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:18,160 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:18,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534477042_0001_00_000003, stoppedTime=1463534478160, exitStatus=0
2016-05-18 01:21:18,160 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:18,160 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:18,161 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534477042_0001_1_00 [F718CBD99C794830A157279577BBF390]
2016-05-18 01:21:18,161 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534477042_0001_1_01 [A7414596E620413586EA78F36907996C]
2016-05-18 01:21:18,161 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534477042_0001_1
2016-05-18 01:21:18,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534477042_0001_1
2016-05-18 01:21:18,161 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534477042_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:18,162 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534477042_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:21:18,166 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C2AF9CA230EA4A8A8A3D486482D42354 for vertex: vertex_1463534477042_0001_1_02 [B7EC8D8BAC9C4806B5FC9426B6832563], outputName: C2AF9CA230EA4A8A8A3D486482D42354
2016-05-18 01:21:18,170 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534477042_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:21:18,170 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534477042_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:18,170 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534477042_0001_00_000003, Exiting
2016-05-18 01:21:18,171 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534477042_0001_00_000003
2016-05-18 01:21:18,171 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534477042_0001_00_000003
2016-05-18 01:21:18,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534477042_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:18,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534477042_0001_00_000003 completed successfully
2016-05-18 01:21:18,178 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C2AF9CA230EA4A8A8A3D486482D42354 of vertex/vertexGroup:B7EC8D8BAC9C4806B5FC9426B6832563 isVertexGroupOutput:false
2016-05-18 01:21:18,178 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:18,182 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534477042_0001_1, queueSize=0
2016-05-18 01:21:18,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534477042_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534477042_0001_1, startTime=1463534477443, finishTime=1463534478178, timeTaken=735, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=1790, AM_GC_TIME_MILLIS=438, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=23, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390603433791, Process_Duration=21, Process_End_Time=4390603433812, Read_Duration=6, Tuples_Read=20, Tuples_Written=15, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-18 01:21:18,183 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534477042_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:18,183 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534477042_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:18 Completed Dag: dag_1463534477042_0001_1
2016-05-18 01:21:18,183 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534477042_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:18,184 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534477042_0001_1_post
2016-05-18 01:21:18,184 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,184 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:18,184 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:18,184 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown, with id=dag_1463534477042_0001_1
2016-05-18 01:21:18,185 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F] lower*upper/(1/1) ...lexcogrouprelativeunknown, with id=dag_1463534477042_0001_1
2016-05-18 01:21:18,195 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1463534477042_0001
2016-05-18 01:21:18,197 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:18,197 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:18,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:18,198 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:18,198 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:18,198 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:18,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:18,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:18,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:18,199 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:18,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:18,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:18,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:18,202 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:18,206 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/1CD5AE5C90B44577B735BFC377C8A5D7/A501DFD6376A484AA12082FC12E87C2F/.tez/application_1463534477042_0001
2016-05-18 01:21:18,206 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown/_temporary
2016-05-18 01:21:18,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:01.189
2016-05-18 01:21:18,291 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-05-18 01:21:18,322 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:18,322 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:18,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:21:18,395 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-18 01:21:18,395 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:18,395 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:18,396 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:18,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:18,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-18 01:21:18,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:18,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:18,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-18 01:21:18,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:18,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:18,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:18,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-05-18 01:21:18,490 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:18,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:18,509 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:18,509 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:18,509 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:18,518 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001 doesn't exist and is created
2016-05-18 01:21:18,537 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:18,537 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:18,538 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd
2016-05-18 01:21:18,554 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534478509_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:18,554 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:18,554 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:18,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:18,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:18,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:18,557 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:18,558 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:18,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:18,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:18,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:18,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:18,560 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:18,561 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:18,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:18,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:18,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:18,561 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:18,563 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:18,563 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:18,565 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:18,565 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:18,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:18,566 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-log-dir
2016-05-18 01:21:18,566 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-log-dir/history.txt.appattempt_1463534478509_0001_000000, maxErrors=10
2016-05-18 01:21:18,566 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:18,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534478509_0001_000000, appSubmitTime=1463534478553, launchTime=1463534478553
2016-05-18 01:21:18,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:18,577 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55843
2016-05-18 01:21:18,577 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534478509
2016-05-18 01:21:18,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534478509_0001_000000, startTime=1463534478579
2016-05-18 01:21:18,579 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:18,637 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:18,638 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:18,638 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463534478509_0001, dagName=[A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative
2016-05-18 01:21:18,657 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative
2016-05-18 01:21:18,657 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:18,658 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534478509_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-log-dir/dag_1463534478509_0001_1.dot
2016-05-18 01:21:18,658 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-log-dir/dag_1463534478509_0001_1-tez-dag.pb.txt
2016-05-18 01:21:18,669 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534478509_0001_1
2016-05-18 01:21:18,669 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:18,669 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative
2016-05-18 01:21:18 Running Dag: dag_1463534478509_0001_1
2016-05-18 01:21:18,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534478509_0001_1, submitTime=1463534478657
2016-05-18 01:21:18,684 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001/tez-conf.pb" } size: 40949 timestamp: 1463534478000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:18,684 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,684 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:18,685 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001/tez-conf.pb" } size: 40949 timestamp: 1463534478000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:18,685 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:18,685 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:18,685 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:18,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534478509_0001_1, initTime=1463534478686
2016-05-18 01:21:18,686 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534478509_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:18,686 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:18,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:DAG_STARTED]: dagID=dag_1463534478509_0001_1, startTime=1463534478687
2016-05-18 01:21:18,687 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534478509_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D7BF932E25604601BEFBB362773B1246 : {86C54CCA108743798EBB44D9596E5202={InputName=86C54CCA108743798EBB44D9596E5202}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 86C54CCA108743798EBB44D9596E5202, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,687 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,688 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:18,688 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 86C54CCA108743798EBB44D9596E5202 on vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,688 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:18,689 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463534478509_0001, dagName=[A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative
2016-05-18 01:21:18,692 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:18,692 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:18,692 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:18,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534478509_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:18,692 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E], numAdditionalOutputs=1
2016-05-18 01:21:18,692 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=C7502E0517F04C2F96A17E5B0E42D0B3, vertex=vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:18,695 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FDE76BD471AE4195AF1BBE2AFB705D3E:C7502E0517F04C2F96A17E5B0E42D0B3 using oldmapred API
2016-05-18 01:21:18,696 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=C7502E0517F04C2F96A17E5B0E42D0B3, vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:18,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, vertexId=vertex_1463534478509_0001_1_01, initRequestedTime=1463534478688, initedTime=1463534478692, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:18,697 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:18,699 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 86C54CCA108743798EBB44D9596E5202 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:18,701 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:18,701 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:18,702 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:18,702 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:18,702 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 86C54CCA108743798EBB44D9596E5202 on vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:18,703 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:18,704 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {86C54CCA108743798EBB44D9596E5202=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:18,704 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] parallelism set to 1
2016-05-18 01:21:19,077 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:19,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D7BF932E25604601BEFBB362773B1246, vertexId=vertex_1463534478509_0001_1_00, initRequestedTime=1463534478687, initedTime=1463534479078, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:19,078 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] with distanceFromRoot: 0
2016-05-18 01:21:19,078 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:19,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534478509_0001_1_00, startRequestedTime=1463534478688, startedTime=1463534479078
2016-05-18 01:21:19,079 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534478509_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:19,079 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:19,079 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534478509_0001_1_00 for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] numStartedSources: 1 numSources: 1
2016-05-18 01:21:19,079 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] with distanceFromRoot: 1
2016-05-18 01:21:19,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534478509_0001_1_01, startRequestedTime=1463534479079, startedTime=1463534479079
2016-05-18 01:21:19,081 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:19,081 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D7BF932E25604601BEFBB362773B1246
2016-05-18 01:21:19,081 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:19,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_STARTED]: vertexName=D7BF932E25604601BEFBB362773B1246, taskId=task_1463534478509_0001_1_00_000000, scheduledTime=1463534479081, launchTime=1463534479081
2016-05-18 01:21:19,082 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:19,083 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: FDE76BD471AE4195AF1BBE2AFB705D3E with 0 source tasks and 1 pending tasks
2016-05-18 01:21:19,084 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D7BF932E25604601BEFBB362773B1246 in vertex: FDE76BD471AE4195AF1BBE2AFB705D3E numBipartiteSourceTasks: 1
2016-05-18 01:21:19,085 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:19,087 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534478509_0001_00_000001, launchTime=1463534479086
2016-05-18 01:21:19,087 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534478509_0001_00_000001
2016-05-18 01:21:19,088 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534478509_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,089 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534478509_0001_1_00_000000_0] started. Is using containerId: [container_1463534478509_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:19,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D7BF932E25604601BEFBB362773B1246, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0, startTime=1463534479088, containerId=container_1463534478509_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:19 Starting to run new task attempt: attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,090 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,090 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,090 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:19,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative, VertexName: D7BF932E25604601BEFBB362773B1246, VertexParallelism: 1, TaskAttemptID:attempt_1463534478509_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86C54CCA108743798EBB44D9596E5202, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:19,091 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:19,091 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534478509_0001, with dag id: dag_1463534478509_0001_1
2016-05-18 01:21:19,092 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,094 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534478509_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:19,094 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534478509_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:19,099 INFO input.MRInput (MRInput.java:initialize(440)) - 86C54CCA108743798EBB44D9596E5202 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:19,099 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 86C54CCA108743798EBB44D9596E5202
2016-05-18 01:21:19,102 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:19,110 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D7BF932E25604601BEFBB362773B1246, ordinal: 0
2016-05-18 01:21:19,110 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7BF932E25604601BEFBB362773B1246, mem on start (mb), free: 349, total: 377, max: 672
2016-05-18 01:21:19,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:19,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:19,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:19,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:19,111 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[86C54CCA108743798EBB44D9596E5202:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [FDE76BD471AE4195AF1BBE2AFB705D3E:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:19,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:19,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:19,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:19,112 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[86C54CCA108743798EBB44D9596E5202:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [FDE76BD471AE4195AF1BBE2AFB705D3E:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:19,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:19,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 86C54CCA108743798EBB44D9596E5202 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:19,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:19,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:19,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 86C54CCA108743798EBB44D9596E5202
2016-05-18 01:21:19,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:19,114 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,117 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:19,118 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 86C54CCA108743798EBB44D9596E5202
2016-05-18 01:21:19,118 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 8556E0DD13BE47CF8B6FD092F160B7FA
2016-05-18 01:21:19,119 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 8556E0DD13BE47CF8B6FD092F160B7FA
2016-05-18 01:21:19,119 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:19,119 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 86C54CCA108743798EBB44D9596E5202 initialized RecordReader from event
2016-05-18 01:21:19,121 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - FDE76BD471AE4195AF1BBE2AFB705D3E using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@180f0b13, comparator=cascading.tuple.hadoop.util.TupleComparator@3a018bc, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:19,121 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for FDE76BD471AE4195AF1BBE2AFB705D3E: , UsingHashComparator=false
2016-05-18 01:21:19,133 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:19,133 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:19,134 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for FDE76BD471AE4195AF1BBE2AFB705D3E: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:19,134 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - FDE76BD471AE4195AF1BBE2AFB705D3E: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:19,135 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:19,135 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D7BF932E25604601BEFBB362773B1246, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:19,139 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,139 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7BF932E25604601BEFBB362773B1246, mem on close (mb), free: 247, total: 377, max: 672
2016-05-18 01:21:19,140 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - FDE76BD471AE4195AF1BBE2AFB705D3E: Starting flush of map output
2016-05-18 01:21:19,140 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - FDE76BD471AE4195AF1BBE2AFB705D3E: Span0.length = 10, perItem = 21
2016-05-18 01:21:19,140 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - FDE76BD471AE4195AF1BBE2AFB705D3E: done sorting span=0, length=10, time=0
2016-05-18 01:21:19,141 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - FDE76BD471AE4195AF1BBE2AFB705D3E: Heap = SpanIterator<0:9> (span=Span[16777216,210]),
2016-05-18 01:21:19,143 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - FDE76BD471AE4195AF1BBE2AFB705D3E: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-local-dir/output/attempt_1463534478509_0001_1_00_000000_0_10034_0/file.out
2016-05-18 01:21:19,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:19 Completed running task attempt: attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,149 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:19,149 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534478509_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:19,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534478509_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534479135, Process_Duration=4, Process_End_Time=1463534479139, Read_Duration=4, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:21:19,151 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534478509_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,151 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,151 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:19,151 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D7BF932E25604601BEFBB362773B1246, taskAttemptId=attempt_1463534478509_0001_1_00_000000_0, creationTime=1463534479082, allocationTime=1463534479085, startTime=1463534479088, finishTime=1463534479151, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:19,152 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534478509_0001_1_00_000000_0
2016-05-18 01:21:19,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_FINISHED]: vertexName=D7BF932E25604601BEFBB362773B1246, taskId=task_1463534478509_0001_1_00_000000, startTime=1463534479088, finishTime=1463534479153, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463534478509_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534479135, Process_Duration=4, Process_End_Time=1463534479139, Read_Duration=4, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:21:19,154 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:19,154 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:19,152 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534478509_0001_00_000001
2016-05-18 01:21:19,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_FINISHED]: vertexName=D7BF932E25604601BEFBB362773B1246, vertexId=vertex_1463534478509_0001_1_00, initRequestedTime=1463534478687, initedTime=1463534479078, startRequestedTime=1463534478688, startedTime=1463534479078, finishTime=1463534479154, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534479135, Process_Duration=4, Process_End_Time=1463534479139, Read_Duration=4, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463534479088, firstTasksToStart=[ task_1463534478509_0001_1_00_000000 ], lastTaskFinishTime=1463534479151, lastTasksToFinish=[ task_1463534478509_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534478509_0001_1_00_000000 ], longestDurationTasks=[ task_1463534478509_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:19,157 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:19,157 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] attempt: attempt_1463534478509_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:19,158 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FDE76BD471AE4195AF1BBE2AFB705D3E
2016-05-18 01:21:19,158 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D7BF932E25604601BEFBB362773B1246, destinationVertexName=FDE76BD471AE4195AF1BBE2AFB705D3E. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:19,158 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:19,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534478509_0001_00_000001, stoppedTime=1463534479159, exitStatus=0
2016-05-18 01:21:19,159 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:19,159 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:19,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_STARTED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, taskId=task_1463534478509_0001_1_01_000000, scheduledTime=1463534479159, launchTime=1463534479159
2016-05-18 01:21:19,160 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534478509_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:19,160 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:19,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534478509_0001_00_000002, launchTime=1463534479161
2016-05-18 01:21:19,162 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534478509_0001_00_000002
2016-05-18 01:21:19,163 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534478509_0001_1_01_000000_0] started. Is using containerId: [container_1463534478509_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:19,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0, startTime=1463534479162, containerId=container_1463534478509_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:19,163 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534478509_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19 Starting to run new task attempt: attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,164 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,164 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,164 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:19,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative, VertexName: FDE76BD471AE4195AF1BBE2AFB705D3E, VertexParallelism: 1, TaskAttemptID:attempt_1463534478509_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7BF932E25604601BEFBB362773B1246, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C7502E0517F04C2F96A17E5B0E42D0B3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:19,165 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:19,165 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534478509_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:19,165 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534478509_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:19,165 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534478509_0001_00_000001, Exiting
2016-05-18 01:21:19,165 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534478509_0001_00_000001
2016-05-18 01:21:19,165 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534478509_0001_00_000001
2016-05-18 01:21:19,166 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,166 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534478509_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:19,167 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534478509_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:19,167 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534478509_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:19,167 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534478509_0001_00_000001 completed successfully
2016-05-18 01:21:19,170 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:19,171 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:19,184 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDE76BD471AE4195AF1BBE2AFB705D3E, ordinal: 1
2016-05-18 01:21:19,184 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDE76BD471AE4195AF1BBE2AFB705D3E, mem on start (mb), free: 242, total: 377, max: 672
2016-05-18 01:21:19,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:19,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:19,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:19,187 INFO output.MROutput (MROutput.java:initialize(421)) - C7502E0517F04C2F96A17E5B0E42D0B3: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:19,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:19,187 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C7502E0517F04C2F96A17E5B0E42D0B3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [D7BF932E25604601BEFBB362773B1246:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:19,187 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:19,188 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:19,188 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:19,188 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C7502E0517F04C2F96A17E5B0E42D0B3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [D7BF932E25604601BEFBB362773B1246:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:19,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:19,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D7BF932E25604601BEFBB362773B1246 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:19,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:19,189 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D7BF932E25604601BEFBB362773B1246: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:19,189 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D7BF932E25604601BEFBB362773B1246: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:19,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:19,190 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D7BF932E25604601BEFBB362773B1246 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:19,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D7BF932E25604601BEFBB362773B1246
2016-05-18 01:21:19,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:19,191 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,192 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D7BF932E25604601BEFBB362773B1246: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:19,193 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534478509_0001_1_00_000000_0_10034} to DISK_DIRECT, csize=204, dsize=200, EndTime=1463534479193, TimeTaken=1, Rate=0.19 MB/s
2016-05-18 01:21:19,194 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D7BF932E25604601BEFBB362773B1246
2016-05-18 01:21:19,194 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:19,195 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D7BF932E25604601BEFBB362773B1246, wasInterrupted=false
2016-05-18 01:21:19,195 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-18 01:21:19,195 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:19,196 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:19,196 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 8556E0DD13BE47CF8B6FD092F160B7FA
2016-05-18 01:21:19,197 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: C7502E0517F04C2F96A17E5B0E42D0B3
2016-05-18 01:21:19,197 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001_wd/localmode-local-dir/output/attempt_1463534478509_0001_1_00_000000_0_10034/file.out, len=204, isLocal=true
2016-05-18 01:21:19,197 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-05-18 01:21:19,197 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 8556E0DD13BE47CF8B6FD092F160B7FA, for 1 inputs
2016-05-18 01:21:19,197 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 204 bytes from disk
2016-05-18 01:21:19,198 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:19,198 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:19,198 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-18 01:21:19,198 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D7BF932E25604601BEFBB362773B1246
2016-05-18 01:21:19,198 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D7BF932E25604601BEFBB362773B1246: Shuffle Runner thread complete
2016-05-18 01:21:19,201 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDE76BD471AE4195AF1BBE2AFB705D3E, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:21:19,202 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D7BF932E25604601BEFBB362773B1246: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:19,206 INFO output.MROutput (MROutput.java:close(526)) - C7502E0517F04C2F96A17E5B0E42D0B3 closed
2016-05-18 01:21:19,207 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534478509_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:19,208 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,208 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDE76BD471AE4195AF1BBE2AFB705D3E, mem on close (mb), free: 238, total: 377, max: 672
2016-05-18 01:21:19,208 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D7BF932E25604601BEFBB362773B1246
2016-05-18 01:21:19,208 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D7BF932E25604601BEFBB362773B1246: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:19,209 INFO output.MROutput (MROutput.java:close(526)) - C7502E0517F04C2F96A17E5B0E42D0B3 closed
2016-05-18 01:21:19,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:19 Completed running task attempt: attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,210 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:19,210 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534478509_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:19,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534478509_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534479201, Process_Duration=7, Process_End_Time=1463534479208, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-18 01:21:19,212 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534478509_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,212 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, taskAttemptId=attempt_1463534478509_0001_1_01_000000_0, creationTime=1463534479159, allocationTime=1463534479161, startTime=1463534479162, finishTime=1463534479212, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:19,213 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:19,213 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,214 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534478509_0001_00_000002
2016-05-18 01:21:19,213 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534478509_0001_1_01_000000_0
2016-05-18 01:21:19,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:TASK_FINISHED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, taskId=task_1463534478509_0001_1_01_000000, startTime=1463534479162, finishTime=1463534479214, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463534478509_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534479201, Process_Duration=7, Process_End_Time=1463534479208, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-18 01:21:19,215 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:19,216 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E]
2016-05-18 01:21:19,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:VERTEX_FINISHED]: vertexName=FDE76BD471AE4195AF1BBE2AFB705D3E, vertexId=vertex_1463534478509_0001_1_01, initRequestedTime=1463534478688, initedTime=1463534478692, startRequestedTime=1463534479079, startedTime=1463534479079, finishTime=1463534479216, timeTaken=137, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534479201, Process_Duration=7, Process_End_Time=1463534479208, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463534479162, firstTasksToStart=[ task_1463534478509_0001_1_01_000000 ], lastTaskFinishTime=1463534479212, lastTasksToFinish=[ task_1463534478509_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534478509_0001_1_01_000000 ], longestDurationTasks=[ task_1463534478509_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:19,217 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:19,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534478509_0001_00_000002, stoppedTime=1463534479217, exitStatus=0
2016-05-18 01:21:19,217 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:19,218 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:19,218 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534478509_0001_1_00 [D7BF932E25604601BEFBB362773B1246]
2016-05-18 01:21:19,218 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534478509_0001_1
2016-05-18 01:21:19,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534478509_0001_1
2016-05-18 01:21:19,218 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: C7502E0517F04C2F96A17E5B0E42D0B3 for vertex: vertex_1463534478509_0001_1_01 [FDE76BD471AE4195AF1BBE2AFB705D3E], outputName: C7502E0517F04C2F96A17E5B0E42D0B3
2016-05-18 01:21:19,218 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534478509_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:19,219 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534478509_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:19,224 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534478509_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:19,224 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534478509_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:19,225 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534478509_0001_00_000002, Exiting
2016-05-18 01:21:19,225 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534478509_0001_00_000002
2016-05-18 01:21:19,225 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534478509_0001_00_000002
2016-05-18 01:21:19,225 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534478509_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:19,225 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534478509_0001_00_000002 completed successfully
2016-05-18 01:21:19,227 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:C7502E0517F04C2F96A17E5B0E42D0B3 of vertex/vertexGroup:FDE76BD471AE4195AF1BBE2AFB705D3E isVertexGroupOutput:false
2016-05-18 01:21:19,228 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:19,230 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534478509_0001_1, queueSize=0
2016-05-18 01:21:19,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534478509_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534478509_0001_1, startTime=1463534478686, finishTime=1463534479228, timeTaken=542, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1280, AM_GC_TIME_MILLIS=372, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068958336, Process_Duration=11, Process_End_Time=2927068958347, Read_Duration=5, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-18 01:21:19,231 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534478509_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:19,231 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534478509_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:19 Completed Dag: dag_1463534478509_0001_1
2016-05-18 01:21:19,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534478509_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:19,232 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534478509_0001_1_post
2016-05-18 01:21:19,232 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:19,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:19,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative, with id=dag_1463534478509_0001_1
2016-05-18 01:21:19,232 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE] test/(1/1) ...leRelative/simplerelative, with id=dag_1463534478509_0001_1
2016-05-18 01:21:19,239 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1463534478509_0001
2016-05-18 01:21:19,241 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:19,241 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:19,241 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:19,241 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:19,241 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:19,242 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:19,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:19,244 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:19,244 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:19,244 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:19,244 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:19,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:19,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:19,246 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:19,249 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/A1BF2FB35CB645D48862AEF9D95E9333/C0E86297875B45EEAE21A085FAF5FEDE/.tez/application_1463534478509_0001
2016-05-18 01:21:19,250 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative/_temporary
2016-05-18 01:21:19,251 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.764
2016-05-18 01:21:19,292 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-05-18 01:21:19,429 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:19,430 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:19,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 01:21:19,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.048
2016-05-18 01:21:19,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:19,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:19,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:19,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:21:19,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-18 01:21:19,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:19,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:19,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-18 01:21:19,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:19,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:19,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:19,613 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-05-18 01:21:19,613 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:19,616 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:19,616 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:19,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:19,633 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:19,634 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:19,634 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:19,642 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001 doesn't exist and is created
2016-05-18 01:21:19,661 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:19,662 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:19,662 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd
2016-05-18 01:21:19,678 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534479633_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:19,678 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:19,678 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:19,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:19,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:19,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:19,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:19,681 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:19,682 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:19,682 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:19,682 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:19,682 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:19,683 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:19,683 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:19,684 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:19,685 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:19,686 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:19,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:19,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-log-dir
2016-05-18 01:21:19,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-log-dir/history.txt.appattempt_1463534479633_0001_000000, maxErrors=10
2016-05-18 01:21:19,687 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:19,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534479633_0001_000000, appSubmitTime=1463534479677, launchTime=1463534479677
2016-05-18 01:21:19,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:19,694 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60833
2016-05-18 01:21:19,694 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534479633
2016-05-18 01:21:19,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534479633_0001_000000, startTime=1463534479698
2016-05-18 01:21:19,698 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:19,762 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:19,762 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:19,763 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463534479633_0001, dagName=[660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult
2016-05-18 01:21:19,782 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult
2016-05-18 01:21:19,782 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:19,783 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534479633_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-log-dir/dag_1463534479633_0001_1.dot
2016-05-18 01:21:19,784 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-log-dir/dag_1463534479633_0001_1-tez-dag.pb.txt
2016-05-18 01:21:19,792 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534479633_0001_1
2016-05-18 01:21:19,792 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,792 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult
2016-05-18 01:21:19 Running Dag: dag_1463534479633_0001_1
2016-05-18 01:21:19,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534479633_0001_1, submitTime=1463534479782
2016-05-18 01:21:19,809 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001/tez-conf.pb" } size: 40949 timestamp: 1463534479000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:19,809 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,809 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:19,810 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001/tez-conf.pb" } size: 40949 timestamp: 1463534479000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:19,810 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,810 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:19,811 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:19,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534479633_0001_1, initTime=1463534479811
2016-05-18 01:21:19,812 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534479633_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:19,812 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:19,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:DAG_STARTED]: dagID=dag_1463534479633_0001_1, startTime=1463534479812
2016-05-18 01:21:19,813 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463534479633_0001, dagName=[660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult
2016-05-18 01:21:19,813 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534479633_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:19,813 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C754E68C4E7F4A07957930D02FCCB8F1 : {3BEDC5380CBC4A88AA71CA40A025DD19={InputName=3BEDC5380CBC4A88AA71CA40A025DD19}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:19,813 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 3BEDC5380CBC4A88AA71CA40A025DD19, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:19,813 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,813 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,813 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,814 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,814 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:19,814 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,816 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 3BEDC5380CBC4A88AA71CA40A025DD19 on vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,818 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 3BEDC5380CBC4A88AA71CA40A025DD19 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:19,818 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:19,818 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,819 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534479633_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:19,821 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF], numAdditionalOutputs=1
2016-05-18 01:21:19,821 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CBAD1DDC4FFA40C6989E8E5D64210BF3, vertex=vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:19,829 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for ADE05164B2464BFEA9862EB8B571F0BF:CBAD1DDC4FFA40C6989E8E5D64210BF3 using oldmapred API
2016-05-18 01:21:19,829 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CBAD1DDC4FFA40C6989E8E5D64210BF3, vertexName=ADE05164B2464BFEA9862EB8B571F0BF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:19,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, vertexId=vertex_1463534479633_0001_1_01, initRequestedTime=1463534479814, initedTime=1463534479821, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:19,831 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:19,831 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:19,831 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:19,831 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:19,831 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:19,832 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 3BEDC5380CBC4A88AA71CA40A025DD19 on vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,833 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:19,833 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {3BEDC5380CBC4A88AA71CA40A025DD19=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:19,834 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] parallelism set to 1
2016-05-18 01:21:19,834 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:19,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, vertexId=vertex_1463534479633_0001_1_00, initRequestedTime=1463534479813, initedTime=1463534479835, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:19,835 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] with distanceFromRoot: 0
2016-05-18 01:21:19,835 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:19,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534479633_0001_1_00, startRequestedTime=1463534479814, startedTime=1463534479836
2016-05-18 01:21:19,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534479633_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:19,836 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:19,837 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534479633_0001_1_00 for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] numStartedSources: 1 numSources: 1
2016-05-18 01:21:19,837 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] with distanceFromRoot: 1
2016-05-18 01:21:19,837 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C754E68C4E7F4A07957930D02FCCB8F1
2016-05-18 01:21:19,838 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534479633_0001_1_01, startRequestedTime=1463534479837, startedTime=1463534479838
2016-05-18 01:21:19,839 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:19,840 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: ADE05164B2464BFEA9862EB8B571F0BF with 0 source tasks and 1 pending tasks
2016-05-18 01:21:19,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_STARTED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, taskId=task_1463534479633_0001_1_00_000000, scheduledTime=1463534479840, launchTime=1463534479840
2016-05-18 01:21:19,841 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,841 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: C754E68C4E7F4A07957930D02FCCB8F1 in vertex: ADE05164B2464BFEA9862EB8B571F0BF numBipartiteSourceTasks: 1
2016-05-18 01:21:19,842 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:19,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534479633_0001_00_000001, launchTime=1463534479843
2016-05-18 01:21:19,844 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534479633_0001_00_000001
2016-05-18 01:21:19,844 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534479633_0001, with dag id: dag_1463534479633_0001_1
2016-05-18 01:21:19,847 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534479633_0001_1_00_000000_0] started. Is using containerId: [container_1463534479633_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:19,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0, startTime=1463534479846, containerId=container_1463534479633_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:19,848 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534479633_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19 Starting to run new task attempt: attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,850 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,851 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,852 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:19,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult, VertexName: C754E68C4E7F4A07957930D02FCCB8F1, VertexParallelism: 1, TaskAttemptID:attempt_1463534479633_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3BEDC5380CBC4A88AA71CA40A025DD19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ADE05164B2464BFEA9862EB8B571F0BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:19,853 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:19,854 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,855 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534479633_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:19,855 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534479633_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:19,861 INFO input.MRInput (MRInput.java:initialize(440)) - 3BEDC5380CBC4A88AA71CA40A025DD19 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:19,861 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 3BEDC5380CBC4A88AA71CA40A025DD19
2016-05-18 01:21:19,862 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:19,872 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C754E68C4E7F4A07957930D02FCCB8F1, ordinal: 0
2016-05-18 01:21:19,872 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C754E68C4E7F4A07957930D02FCCB8F1, mem on start (mb), free: 339, total: 376, max: 672
2016-05-18 01:21:19,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:19,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:19,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:19,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:19,873 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[3BEDC5380CBC4A88AA71CA40A025DD19:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [ADE05164B2464BFEA9862EB8B571F0BF:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:19,874 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:19,874 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:19,874 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:19,875 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[3BEDC5380CBC4A88AA71CA40A025DD19:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [ADE05164B2464BFEA9862EB8B571F0BF:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:19,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:19,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 3BEDC5380CBC4A88AA71CA40A025DD19 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:19,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 3BEDC5380CBC4A88AA71CA40A025DD19
2016-05-18 01:21:19,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:19,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:19,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:19,877 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,879 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:19,879 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:19,879 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 3BEDC5380CBC4A88AA71CA40A025DD19 initialized RecordReader from event
2016-05-18 01:21:19,880 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: 3BEDC5380CBC4A88AA71CA40A025DD19
2016-05-18 01:21:19,880 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: D598E95F855742CEAC21ED498EB36FA9
2016-05-18 01:21:19,881 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] D598E95F855742CEAC21ED498EB36FA9
2016-05-18 01:21:19,884 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - ADE05164B2464BFEA9862EB8B571F0BF using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@419ca976, comparator=cascading.tuple.hadoop.util.TupleComparator@1439a3a1, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:19,885 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for ADE05164B2464BFEA9862EB8B571F0BF: , UsingHashComparator=false
2016-05-18 01:21:19,898 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:19,898 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:19,898 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for ADE05164B2464BFEA9862EB8B571F0BF: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:19,899 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - ADE05164B2464BFEA9862EB8B571F0BF: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:19,975 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:19,976 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C754E68C4E7F4A07957930D02FCCB8F1, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:19,979 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,979 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C754E68C4E7F4A07957930D02FCCB8F1, mem on close (mb), free: 232, total: 376, max: 672
2016-05-18 01:21:19,980 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - ADE05164B2464BFEA9862EB8B571F0BF: Starting flush of map output
2016-05-18 01:21:19,980 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - ADE05164B2464BFEA9862EB8B571F0BF: Span0.length = 2, perItem = 11
2016-05-18 01:21:19,980 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - ADE05164B2464BFEA9862EB8B571F0BF: done sorting span=0, length=2, time=0
2016-05-18 01:21:19,981 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - ADE05164B2464BFEA9862EB8B571F0BF: Heap = SpanIterator<0:1> (span=Span[16777216,22]),
2016-05-18 01:21:19,982 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - ADE05164B2464BFEA9862EB8B571F0BF: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-local-dir/output/attempt_1463534479633_0001_1_00_000000_0_10040_0/file.out
2016-05-18 01:21:19,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:19 Completed running task attempt: attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,986 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:19,986 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534479633_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:19,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534479633_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534479976, Process_Duration=3, Process_End_Time=1463534479979, Read_Duration=0, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:19,987 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534479633_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,988 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:19,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, taskAttemptId=attempt_1463534479633_0001_1_00_000000_0, creationTime=1463534479841, allocationTime=1463534479843, startTime=1463534479846, finishTime=1463534479987, timeTaken=141, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:19,989 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534479633_0001_1_00_000000_0
2016-05-18 01:21:19,988 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:19,989 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:19,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_FINISHED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, taskId=task_1463534479633_0001_1_00_000000, startTime=1463534479846, finishTime=1463534479989, timeTaken=143, status=SUCCEEDED, successfulAttemptID=attempt_1463534479633_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534479976, Process_Duration=3, Process_End_Time=1463534479979, Read_Duration=0, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:19,991 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534479633_0001_00_000001
2016-05-18 01:21:19,991 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:19,991 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:19,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_FINISHED]: vertexName=C754E68C4E7F4A07957930D02FCCB8F1, vertexId=vertex_1463534479633_0001_1_00, initRequestedTime=1463534479813, initedTime=1463534479835, startRequestedTime=1463534479814, startedTime=1463534479836, finishTime=1463534479991, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534479976, Process_Duration=3, Process_End_Time=1463534479979, Read_Duration=0, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534479846, firstTasksToStart=[ task_1463534479633_0001_1_00_000000 ], lastTaskFinishTime=1463534479987, lastTasksToFinish=[ task_1463534479633_0001_1_00_000000 ], minTaskDuration=141, maxTaskDuration=141, avgTaskDuration=141.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534479633_0001_1_00_000000 ], longestDurationTasks=[ task_1463534479633_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:19,993 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:19,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534479633_0001_00_000001, stoppedTime=1463534479993, exitStatus=0
2016-05-18 01:21:19,993 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] attempt: attempt_1463534479633_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:19,993 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ADE05164B2464BFEA9862EB8B571F0BF
2016-05-18 01:21:19,993 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=C754E68C4E7F4A07957930D02FCCB8F1, destinationVertexName=ADE05164B2464BFEA9862EB8B571F0BF. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:19,994 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,994 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:19,994 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:19,994 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534479633_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:19,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_STARTED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, taskId=task_1463534479633_0001_1_01_000000, scheduledTime=1463534479994, launchTime=1463534479994
2016-05-18 01:21:19,995 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:19,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534479633_0001_00_000002, launchTime=1463534479997
2016-05-18 01:21:20,000 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534479633_0001_00_000002
2016-05-18 01:21:20,000 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534479633_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,001 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534479633_0001_1_01_000000_0] started. Is using containerId: [container_1463534479633_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:20,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0, startTime=1463534480000, containerId=container_1463534479633_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:20 Starting to run new task attempt: attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,001 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,001 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,001 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:20,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult, VertexName: ADE05164B2464BFEA9862EB8B571F0BF, VertexParallelism: 1, TaskAttemptID:attempt_1463534479633_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C754E68C4E7F4A07957930D02FCCB8F1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CBAD1DDC4FFA40C6989E8E5D64210BF3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:20,002 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:20,002 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,005 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534479633_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:20,005 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534479633_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:20,005 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534479633_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:20,005 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534479633_0001_00_000001: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:20,006 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534479633_0001_00_000001, Exiting
2016-05-18 01:21:20,007 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534479633_0001_00_000001
2016-05-18 01:21:20,007 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534479633_0001_00_000001
2016-05-18 01:21:20,011 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534479633_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:20,011 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534479633_0001_00_000001 completed successfully
2016-05-18 01:21:20,012 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:20,016 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:20,023 INFO output.MROutput (MROutput.java:initialize(421)) - CBAD1DDC4FFA40C6989E8E5D64210BF3: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:20,024 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADE05164B2464BFEA9862EB8B571F0BF, ordinal: 1
2016-05-18 01:21:20,025 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADE05164B2464BFEA9862EB8B571F0BF, mem on start (mb), free: 226, total: 376, max: 672
2016-05-18 01:21:20,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:20,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:20,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:20,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:20,026 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[CBAD1DDC4FFA40C6989E8E5D64210BF3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [C754E68C4E7F4A07957930D02FCCB8F1:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:20,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:20,027 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:20,027 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:20,027 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[CBAD1DDC4FFA40C6989E8E5D64210BF3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [C754E68C4E7F4A07957930D02FCCB8F1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:20,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:20,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C754E68C4E7F4A07957930D02FCCB8F1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:20,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:20,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:20,028 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - C754E68C4E7F4A07957930D02FCCB8F1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:20,029 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - C754E68C4E7F4A07957930D02FCCB8F1: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:20,030 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: C754E68C4E7F4A07957930D02FCCB8F1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:20,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C754E68C4E7F4A07957930D02FCCB8F1
2016-05-18 01:21:20,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:20,033 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,034 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C754E68C4E7F4A07957930D02FCCB8F1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:20,036 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534479633_0001_1_00_000000_0_10040} to DISK_DIRECT, csize=36, dsize=32, EndTime=1463534480035, TimeTaken=1, Rate=0.03 MB/s
2016-05-18 01:21:20,036 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : C754E68C4E7F4A07957930D02FCCB8F1
2016-05-18 01:21:20,036 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:20,036 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: C754E68C4E7F4A07957930D02FCCB8F1, wasInterrupted=false
2016-05-18 01:21:20,036 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - C754E68C4E7F4A07957930D02FCCB8F1: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:20,037 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:20,036 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-18 01:21:20,038 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:20,038 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: D598E95F855742CEAC21ED498EB36FA9
2016-05-18 01:21:20,039 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: CBAD1DDC4FFA40C6989E8E5D64210BF3
2016-05-18 01:21:20,040 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001_wd/localmode-local-dir/output/attempt_1463534479633_0001_1_00_000000_0_10040/file.out, len=36, isLocal=true
2016-05-18 01:21:20,040 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-05-18 01:21:20,040 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 36 bytes from disk
2016-05-18 01:21:20,041 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] D598E95F855742CEAC21ED498EB36FA9, for 1 inputs
2016-05-18 01:21:20,041 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:20,041 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:20,042 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-05-18 01:21:20,042 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : C754E68C4E7F4A07957930D02FCCB8F1
2016-05-18 01:21:20,042 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADE05164B2464BFEA9862EB8B571F0BF, all 1 inputs ready in: 00:00:00.001
2016-05-18 01:21:20,042 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - C754E68C4E7F4A07957930D02FCCB8F1: Shuffle Runner thread complete
2016-05-18 01:21:20,043 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - C754E68C4E7F4A07957930D02FCCB8F1: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:20,046 INFO output.MROutput (MROutput.java:close(526)) - CBAD1DDC4FFA40C6989E8E5D64210BF3 closed
2016-05-18 01:21:20,046 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534479633_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:20,047 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,047 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADE05164B2464BFEA9862EB8B571F0BF, mem on close (mb), free: 223, total: 376, max: 672
2016-05-18 01:21:20,047 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: C754E68C4E7F4A07957930D02FCCB8F1
2016-05-18 01:21:20,047 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C754E68C4E7F4A07957930D02FCCB8F1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:20,048 INFO output.MROutput (MROutput.java:close(526)) - CBAD1DDC4FFA40C6989E8E5D64210BF3 closed
2016-05-18 01:21:20,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:20 Completed running task attempt: attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,048 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:20,048 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534479633_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:20,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534479633_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=2, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534480042, Process_Duration=5, Process_End_Time=1463534480047, Read_Duration=1, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-05-18 01:21:20,050 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534479633_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:20,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, taskAttemptId=attempt_1463534479633_0001_1_01_000000_0, creationTime=1463534479995, allocationTime=1463534479996, startTime=1463534480000, finishTime=1463534480050, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:20,050 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:20,050 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:20,050 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534479633_0001_1_01_000000_0
2016-05-18 01:21:20,051 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:TASK_FINISHED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, taskId=task_1463534479633_0001_1_01_000000, startTime=1463534480000, finishTime=1463534480051, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463534479633_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=2, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534480042, Process_Duration=5, Process_End_Time=1463534480047, Read_Duration=1, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-05-18 01:21:20,052 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:20,052 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF]
2016-05-18 01:21:20,053 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534479633_0001_00_000002
2016-05-18 01:21:20,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:VERTEX_FINISHED]: vertexName=ADE05164B2464BFEA9862EB8B571F0BF, vertexId=vertex_1463534479633_0001_1_01, initRequestedTime=1463534479814, initedTime=1463534479821, startRequestedTime=1463534479837, startedTime=1463534479838, finishTime=1463534480052, timeTaken=214, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=2, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534480042, Process_Duration=5, Process_End_Time=1463534480047, Read_Duration=1, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1463534480000, firstTasksToStart=[ task_1463534479633_0001_1_01_000000 ], lastTaskFinishTime=1463534480050, lastTasksToFinish=[ task_1463534479633_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534479633_0001_1_01_000000 ], longestDurationTasks=[ task_1463534479633_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:20,055 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:20,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534479633_0001_00_000002, stoppedTime=1463534480056, exitStatus=0
2016-05-18 01:21:20,056 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:20,056 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:20,056 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534479633_0001_1_00 [C754E68C4E7F4A07957930D02FCCB8F1]
2016-05-18 01:21:20,056 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534479633_0001_1
2016-05-18 01:21:20,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534479633_0001_1
2016-05-18 01:21:20,057 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CBAD1DDC4FFA40C6989E8E5D64210BF3 for vertex: vertex_1463534479633_0001_1_01 [ADE05164B2464BFEA9862EB8B571F0BF], outputName: CBAD1DDC4FFA40C6989E8E5D64210BF3
2016-05-18 01:21:20,057 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534479633_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:20,057 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534479633_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:20,065 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534479633_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:20,065 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534479633_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:20,066 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534479633_0001_00_000002, Exiting
2016-05-18 01:21:20,066 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534479633_0001_00_000002
2016-05-18 01:21:20,066 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CBAD1DDC4FFA40C6989E8E5D64210BF3 of vertex/vertexGroup:ADE05164B2464BFEA9862EB8B571F0BF isVertexGroupOutput:false
2016-05-18 01:21:20,066 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:20,066 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534479633_0001_00_000002
2016-05-18 01:21:20,068 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534479633_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:20,068 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534479633_0001_00_000002 completed successfully
2016-05-18 01:21:20,069 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534479633_0001_1, queueSize=0
2016-05-18 01:21:20,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534479633_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534479633_0001_1, startTime=1463534479812, finishTime=1463534480066, timeTaken=254, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=430, AM_GC_TIME_MILLIS=5, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=2, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068960018, Process_Duration=8, Process_End_Time=2927068960026, Read_Duration=1, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-05-18 01:21:20,070 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534479633_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:20,070 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534479633_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:20 Completed Dag: dag_1463534479633_0001_1
2016-05-18 01:21:20,071 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534479633_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:20,071 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534479633_0001_1_post
2016-05-18 01:21:20,072 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,072 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:20,072 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:20,073 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult, with id=dag_1463534479633_0001_1
2016-05-18 01:21:20,073 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13] test/(1/1) ...SimpleResult/simpleresult, with id=dag_1463534479633_0001_1
2016-05-18 01:21:20,077 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1463534479633_0001
2016-05-18 01:21:20,079 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:20,079 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:20,079 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:20,079 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:20,079 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:20,081 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:20,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:20,081 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:20,081 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:20,082 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:20,082 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:20,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:20,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:20,083 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:20,086 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/660A8C4746E34CD6AC4F76151224CEDA/FCDB99DBC51C4F21ACF740CEBB0C7D13/.tez/application_1463534479633_0001
2016-05-18 01:21:20,087 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult/_temporary
2016-05-18 01:21:20,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.476
2016-05-18 01:21:20,126 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-05-18 01:21:20,155 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:20,156 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:20,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 01:21:20,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 01:21:20,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:20,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:20,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:20,229 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:20,230 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-18 01:21:20,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:20,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:20,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-18 01:21:20,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:20,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:20,243 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:20,244 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-05-18 01:21:20,244 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:20,262 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:20,266 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:20,267 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:20,267 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:20,275 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001 doesn't exist and is created
2016-05-18 01:21:20,296 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:20,297 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:20,297 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001_wd
2016-05-18 01:21:20,313 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534480267_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:20,313 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:20,314 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:20,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:20,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:20,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:20,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:20,316 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:20,318 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:20,318 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:20,319 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:20,319 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:20,319 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:20,320 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:20,320 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:20,322 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:20,323 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:20,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:20,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001_wd/localmode-log-dir
2016-05-18 01:21:20,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001_wd/localmode-log-dir/history.txt.appattempt_1463534480267_0001_000000, maxErrors=10
2016-05-18 01:21:20,324 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:20,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534480267_0001_000000, appSubmitTime=1463534480312, launchTime=1463534480312
2016-05-18 01:21:20,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:20,332 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49577
2016-05-18 01:21:20,333 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534480267
2016-05-18 01:21:20,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534480267_0001_000000, startTime=1463534480335
2016-05-18 01:21:20,335 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:20,397 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:20,397 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:20,397 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463534480267_0001, dagName=[8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall
2016-05-18 01:21:20,417 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall
2016-05-18 01:21:20,417 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:20,418 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534480267_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001_wd/localmode-log-dir/dag_1463534480267_0001_1.dot
2016-05-18 01:21:20,419 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001_wd/localmode-log-dir/dag_1463534480267_0001_1-tez-dag.pb.txt
2016-05-18 01:21:20,424 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534480267_0001_1
2016-05-18 01:21:20,424 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,424 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall
2016-05-18 01:21:20 Running Dag: dag_1463534480267_0001_1
2016-05-18 01:21:20,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534480267_0001_1, submitTime=1463534480417
2016-05-18 01:21:20,440 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001/tez-conf.pb" } size: 40949 timestamp: 1463534480000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:20,441 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,441 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,441 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:20,441 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:20,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534480267_0001_1, initTime=1463534480442
2016-05-18 01:21:20,442 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480267_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:20,442 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:20,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:DAG_STARTED]: dagID=dag_1463534480267_0001_1, startTime=1463534480443
2016-05-18 01:21:20,443 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480267_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:20,443 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FA3A345525EA48C3B38C093947269E87 : {5146C8153A58430AADEB31C712007BD2={InputName=5146C8153A58430AADEB31C712007BD2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:20,443 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 5146C8153A58430AADEB31C712007BD2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:20,443 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,443 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,444 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,444 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,445 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:20,445 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 5146C8153A58430AADEB31C712007BD2 on vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,445 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463534480267_0001, dagName=[8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall
2016-05-18 01:21:20,447 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 5146C8153A58430AADEB31C712007BD2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:20,449 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:20,449 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:20,450 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:20,450 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:20,450 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 5146C8153A58430AADEB31C712007BD2 on vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,451 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:20,451 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {5146C8153A58430AADEB31C712007BD2=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:20,452 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] parallelism set to 1
2016-05-18 01:21:20,452 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:20,454 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87], numAdditionalOutputs=1
2016-05-18 01:21:20,454 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D92FCE4F6DED4EE8BF51894D72A1A496, vertex=vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:20,456 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FA3A345525EA48C3B38C093947269E87:D92FCE4F6DED4EE8BF51894D72A1A496 using oldmapred API
2016-05-18 01:21:20,456 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D92FCE4F6DED4EE8BF51894D72A1A496, vertexName=FA3A345525EA48C3B38C093947269E87, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:20,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA3A345525EA48C3B38C093947269E87, vertexId=vertex_1463534480267_0001_1_00, initRequestedTime=1463534480443, initedTime=1463534480454, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:20,457 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] with distanceFromRoot: 0
2016-05-18 01:21:20,457 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:20,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534480267_0001_1_00, startRequestedTime=1463534480445, startedTime=1463534480457
2016-05-18 01:21:20,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534480267_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:20,458 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:20,458 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FA3A345525EA48C3B38C093947269E87
2016-05-18 01:21:20,459 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:TASK_STARTED]: vertexName=FA3A345525EA48C3B38C093947269E87, taskId=task_1463534480267_0001_1_00_000000, scheduledTime=1463534480459, launchTime=1463534480459
2016-05-18 01:21:20,460 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,460 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:20,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534480267_0001_00_000001, launchTime=1463534480462
2016-05-18 01:21:20,463 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480267_0001_00_000001
2016-05-18 01:21:20,463 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534480267_0001_1_00_000000_0] started. Is using containerId: [container_1463534480267_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:20,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA3A345525EA48C3B38C093947269E87, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0, startTime=1463534480463, containerId=container_1463534480267_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:20,466 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480267_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20 Starting to run new task attempt: attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,467 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:20,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall, VertexName: FA3A345525EA48C3B38C093947269E87, VertexParallelism: 1, TaskAttemptID:attempt_1463534480267_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5146C8153A58430AADEB31C712007BD2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D92FCE4F6DED4EE8BF51894D72A1A496, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:20,468 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:20,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534480267_0001, with dag id: dag_1463534480267_0001_1
2016-05-18 01:21:20,469 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,476 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534480267_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:20,476 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534480267_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:20,479 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:20,482 INFO input.MRInput (MRInput.java:initialize(440)) - 5146C8153A58430AADEB31C712007BD2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:20,482 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 5146C8153A58430AADEB31C712007BD2
2016-05-18 01:21:20,488 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA3A345525EA48C3B38C093947269E87, ordinal: 0
2016-05-18 01:21:20,488 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA3A345525EA48C3B38C093947269E87, mem on start (mb), free: 146, total: 376, max: 672
2016-05-18 01:21:20,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:20,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:20,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:20,493 INFO output.MROutput (MROutput.java:initialize(421)) - D92FCE4F6DED4EE8BF51894D72A1A496: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:20,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:20,493 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[5146C8153A58430AADEB31C712007BD2:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D92FCE4F6DED4EE8BF51894D72A1A496:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:20,493 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:20,493 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:20,493 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:21:20,494 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[5146C8153A58430AADEB31C712007BD2:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D92FCE4F6DED4EE8BF51894D72A1A496:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 5146C8153A58430AADEB31C712007BD2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 5146C8153A58430AADEB31C712007BD2
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:20,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:20,495 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,496 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:20,496 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 5146C8153A58430AADEB31C712007BD2 initialized RecordReader from event
2016-05-18 01:21:20,498 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:20,498 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 5146C8153A58430AADEB31C712007BD2
2016-05-18 01:21:20,498 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: D92FCE4F6DED4EE8BF51894D72A1A496
2016-05-18 01:21:20,499 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-05-18 01:21:20,499 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:20,499 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA3A345525EA48C3B38C093947269E87, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:20,501 INFO output.MROutput (MROutput.java:close(526)) - D92FCE4F6DED4EE8BF51894D72A1A496 closed
2016-05-18 01:21:20,501 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534480267_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:21:20,502 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,502 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA3A345525EA48C3B38C093947269E87, mem on close (mb), free: 144, total: 376, max: 672
2016-05-18 01:21:20,503 INFO output.MROutput (MROutput.java:close(526)) - D92FCE4F6DED4EE8BF51894D72A1A496 closed
2016-05-18 01:21:20,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:20 Completed running task attempt: attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,504 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:20,504 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534480267_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:20,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534480267_0001_1_00_000000_0: Counters: 8 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463534480499, Process_Duration=3, Process_End_Time=1463534480502, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:21:20,506 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534480267_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:20,506 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:20,506 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:20,506 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA3A345525EA48C3B38C093947269E87, taskAttemptId=attempt_1463534480267_0001_1_00_000000_0, creationTime=1463534480460, allocationTime=1463534480461, startTime=1463534480463, finishTime=1463534480506, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:20,507 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534480267_0001_1_00_000000_0
2016-05-18 01:21:20,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:TASK_FINISHED]: vertexName=FA3A345525EA48C3B38C093947269E87, taskId=task_1463534480267_0001_1_00_000000, startTime=1463534480463, finishTime=1463534480507, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463534480267_0001_1_00_000000_0, diagnostics=, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534480499, Process_Duration=3, Process_End_Time=1463534480502, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:21:20,508 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:20,508 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87]
2016-05-18 01:21:20,507 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480267_0001_00_000001
2016-05-18 01:21:20,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:VERTEX_FINISHED]: vertexName=FA3A345525EA48C3B38C093947269E87, vertexId=vertex_1463534480267_0001_1_00, initRequestedTime=1463534480443, initedTime=1463534480454, startRequestedTime=1463534480445, startedTime=1463534480457, finishTime=1463534480508, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 9, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534480499, Process_Duration=3, Process_End_Time=1463534480502, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463534480463, firstTasksToStart=[ task_1463534480267_0001_1_00_000000 ], lastTaskFinishTime=1463534480506, lastTasksToFinish=[ task_1463534480267_0001_1_00_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534480267_0001_1_00_000000 ], longestDurationTasks=[ task_1463534480267_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:20,510 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:20,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534480267_0001_00_000001, stoppedTime=1463534480510, exitStatus=0
2016-05-18 01:21:20,510 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:21:20,511 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:20,511 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534480267_0001_1
2016-05-18 01:21:20,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534480267_0001_1
2016-05-18 01:21:20,511 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480267_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:20,511 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D92FCE4F6DED4EE8BF51894D72A1A496 for vertex: vertex_1463534480267_0001_1_00 [FA3A345525EA48C3B38C093947269E87], outputName: D92FCE4F6DED4EE8BF51894D72A1A496
2016-05-18 01:21:20,511 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534480267_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:20,512 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480267_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:20,512 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534480267_0001_00_000001, Exiting
2016-05-18 01:21:20,512 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534480267_0001_00_000001
2016-05-18 01:21:20,512 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534480267_0001_00_000001
2016-05-18 01:21:20,513 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534480267_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:20,513 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534480267_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:20,514 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534480267_0001_00_000001 completed successfully
2016-05-18 01:21:20,521 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D92FCE4F6DED4EE8BF51894D72A1A496 of vertex/vertexGroup:FA3A345525EA48C3B38C093947269E87 isVertexGroupOutput:false
2016-05-18 01:21:20,521 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:20,523 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534480267_0001_1, queueSize=0
2016-05-18 01:21:20,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480267_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534480267_0001_1, startTime=1463534480442, finishTime=1463534480522, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 13, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=130, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534480499, Process_Duration=3, Process_End_Time=1463534480502, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:21:20,523 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534480267_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:20,523 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480267_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:20 Completed Dag: dag_1463534480267_0001_1
2016-05-18 01:21:20,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534480267_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:20,524 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534480267_0001_1_post
2016-05-18 01:21:20,524 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:20,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:20,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall, with id=dag_1463534480267_0001_1
2016-05-18 01:21:20,524 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4] test/(1/1) ...m/testFilterAll/filterall, with id=dag_1463534480267_0001_1
2016-05-18 01:21:20,529 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:20,530 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:20,532 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1463534480267_0001
2016-05-18 01:21:20,534 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:20,534 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:20,534 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:20,534 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:20,534 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:20,535 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:20,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:20,535 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:20,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:20,536 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:20,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:20,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:20,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:20,537 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:20,540 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/8BDA09B2C54147D79B389220EFBD6170/D509AA4D1D2040B9A95030D6820C25F4/.tez/application_1463534480267_0001
2016-05-18 01:21:20,541 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall/_temporary
2016-05-18 01:21:20,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.299
2016-05-18 01:21:20,587 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-05-18 01:21:20,616 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:20,616 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:20,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 01:21:20,672 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 01:21:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:20,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:21:20,697 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-18 01:21:20,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:20,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:20,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-18 01:21:20,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:20,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:20,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:20,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-05-18 01:21:20,753 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:20,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:20,772 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:20,772 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:20,772 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:20,781 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001 doesn't exist and is created
2016-05-18 01:21:20,802 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:20,802 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:20,803 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd
2016-05-18 01:21:20,817 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534480772_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:20,817 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:20,818 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:20,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:20,820 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:20,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:20,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:20,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:20,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:20,823 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:20,823 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:20,823 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:20,823 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:20,823 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:20,824 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:20,824 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:20,824 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:20,826 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:20,827 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:20,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:20,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-log-dir
2016-05-18 01:21:20,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-log-dir/history.txt.appattempt_1463534480772_0001_000000, maxErrors=10
2016-05-18 01:21:20,828 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:20,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534480772_0001_000000, appSubmitTime=1463534480817, launchTime=1463534480817
2016-05-18 01:21:20,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:20,843 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52756
2016-05-18 01:21:20,843 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534480772
2016-05-18 01:21:20,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534480772_0001_000000, startTime=1463534480845
2016-05-18 01:21:20,846 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:20,903 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:20,903 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:20,903 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463534480772_0001, dagName=[04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown
2016-05-18 01:21:20,924 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown
2016-05-18 01:21:20,924 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:20,925 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534480772_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-log-dir/dag_1463534480772_0001_1.dot
2016-05-18 01:21:20,926 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-log-dir/dag_1463534480772_0001_1-tez-dag.pb.txt
2016-05-18 01:21:20,940 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534480772_0001_1
2016-05-18 01:21:20,940 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,941 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown
2016-05-18 01:21:20 Running Dag: dag_1463534480772_0001_1
2016-05-18 01:21:20,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534480772_0001_1, submitTime=1463534480924
2016-05-18 01:21:20,961 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001/tez-conf.pb" } size: 40949 timestamp: 1463534480000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:20,961 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,961 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:20,962 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001/tez-conf.pb" } size: 40949 timestamp: 1463534480000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:20,962 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:20,963 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:20,963 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:20,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534480772_0001_1, initTime=1463534480963
2016-05-18 01:21:20,963 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480772_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:20,964 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:20,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:DAG_STARTED]: dagID=dag_1463534480772_0001_1, startTime=1463534480964
2016-05-18 01:21:20,964 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463534480772_0001, dagName=[04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown
2016-05-18 01:21:20,964 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480772_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:20,965 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E56862D5128D436BAF874D56CB33CBF3 : {EEEEFC1488A44C34980365AF6BBAD015={InputName=EEEEFC1488A44C34980365AF6BBAD015}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:20,965 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: EEEEFC1488A44C34980365AF6BBAD015, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:20,965 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,965 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,965 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,966 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,967 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:20,967 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:20,969 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: EEEEFC1488A44C34980365AF6BBAD015 on vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,971 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:20,971 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:20,971 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input EEEEFC1488A44C34980365AF6BBAD015 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:20,971 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:20,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534480772_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:20,972 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2], numAdditionalOutputs=1
2016-05-18 01:21:20,973 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=9CAC164F703F40A3A11957473A77968A, vertex=vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:20,975 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:20,975 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:20,975 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:20,975 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:20,976 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: EEEEFC1488A44C34980365AF6BBAD015 on vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,976 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C584DC45C2D14ECD888DC859BD65CFA2:9CAC164F703F40A3A11957473A77968A using oldmapred API
2016-05-18 01:21:20,977 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=9CAC164F703F40A3A11957473A77968A, vertexName=C584DC45C2D14ECD888DC859BD65CFA2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:20,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, vertexId=vertex_1463534480772_0001_1_01, initRequestedTime=1463534480967, initedTime=1463534480972, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:20,979 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:20,980 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:20,981 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {EEEEFC1488A44C34980365AF6BBAD015=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:20,982 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] parallelism set to 1
2016-05-18 01:21:20,982 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:20,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, vertexId=vertex_1463534480772_0001_1_00, initRequestedTime=1463534480965, initedTime=1463534480983, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:20,983 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] with distanceFromRoot: 0
2016-05-18 01:21:20,983 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:20,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534480772_0001_1_00, startRequestedTime=1463534480967, startedTime=1463534480983
2016-05-18 01:21:20,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534480772_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:20,985 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:20,985 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E56862D5128D436BAF874D56CB33CBF3
2016-05-18 01:21:20,985 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,985 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534480772_0001_1_00 for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] numStartedSources: 1 numSources: 1
2016-05-18 01:21:20,986 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] with distanceFromRoot: 1
2016-05-18 01:21:20,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534480772_0001_1_01, startRequestedTime=1463534480985, startedTime=1463534480986
2016-05-18 01:21:20,987 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:20,988 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: C584DC45C2D14ECD888DC859BD65CFA2 with 0 source tasks and 1 pending tasks
2016-05-18 01:21:20,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_STARTED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, taskId=task_1463534480772_0001_1_00_000000, scheduledTime=1463534480987, launchTime=1463534480987
2016-05-18 01:21:20,989 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:20,989 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E56862D5128D436BAF874D56CB33CBF3 in vertex: C584DC45C2D14ECD888DC859BD65CFA2 numBipartiteSourceTasks: 1
2016-05-18 01:21:20,990 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534480772_0001, with dag id: dag_1463534480772_0001_1
2016-05-18 01:21:20,991 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:20,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534480772_0001_00_000001, launchTime=1463534480993
2016-05-18 01:21:20,994 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480772_0001_00_000001
2016-05-18 01:21:20,995 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534480772_0001_1_00_000000_0] started. Is using containerId: [container_1463534480772_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:20,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0, startTime=1463534480995, containerId=container_1463534480772_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:20,996 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480772_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:20 Starting to run new task attempt: attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:20,997 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:20,997 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:20,998 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:20,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown, VertexName: E56862D5128D436BAF874D56CB33CBF3, VertexParallelism: 1, TaskAttemptID:attempt_1463534480772_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEEEFC1488A44C34980365AF6BBAD015, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C584DC45C2D14ECD888DC859BD65CFA2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:20,998 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:20,999 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:21,002 INFO input.MRInput (MRInput.java:initialize(440)) - EEEEFC1488A44C34980365AF6BBAD015 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:21,003 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: EEEEFC1488A44C34980365AF6BBAD015
2016-05-18 01:21:21,003 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:21,003 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534480772_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:21,004 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534480772_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:21,009 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E56862D5128D436BAF874D56CB33CBF3, ordinal: 0
2016-05-18 01:21:21,009 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E56862D5128D436BAF874D56CB33CBF3, mem on start (mb), free: 259, total: 376, max: 672
2016-05-18 01:21:21,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:21,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:21,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:21,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:21,010 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[EEEEFC1488A44C34980365AF6BBAD015:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [C584DC45C2D14ECD888DC859BD65CFA2:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:21,010 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:21,011 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:21,011 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:21,011 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[EEEEFC1488A44C34980365AF6BBAD015:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [C584DC45C2D14ECD888DC859BD65CFA2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:21,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:21,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: EEEEFC1488A44C34980365AF6BBAD015 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:21,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:21,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: EEEEFC1488A44C34980365AF6BBAD015
2016-05-18 01:21:21,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:21,013 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:21,015 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:21,015 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - EEEEFC1488A44C34980365AF6BBAD015 initialized RecordReader from event
2016-05-18 01:21:21,017 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:21,017 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: EEEEFC1488A44C34980365AF6BBAD015
2016-05-18 01:21:21,017 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: D110C49F228A441FAC07E933DF387805
2016-05-18 01:21:21,018 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] D110C49F228A441FAC07E933DF387805
2016-05-18 01:21:21,019 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - C584DC45C2D14ECD888DC859BD65CFA2 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5dae927c, comparator=cascading.tuple.hadoop.util.TupleComparator@70133af0, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:21,020 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for C584DC45C2D14ECD888DC859BD65CFA2: , UsingHashComparator=false
2016-05-18 01:21:21,034 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:21,034 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:21,034 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for C584DC45C2D14ECD888DC859BD65CFA2: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:21,034 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - C584DC45C2D14ECD888DC859BD65CFA2: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:21,035 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:21,035 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E56862D5128D436BAF874D56CB33CBF3, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:21,038 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:21,038 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E56862D5128D436BAF874D56CB33CBF3, mem on close (mb), free: 157, total: 376, max: 672
2016-05-18 01:21:21,039 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - C584DC45C2D14ECD888DC859BD65CFA2: Starting flush of map output
2016-05-18 01:21:21,039 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - C584DC45C2D14ECD888DC859BD65CFA2: Span0.length = 5, perItem = 15
2016-05-18 01:21:21,039 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - C584DC45C2D14ECD888DC859BD65CFA2: done sorting span=0, length=5, time=0
2016-05-18 01:21:21,040 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - C584DC45C2D14ECD888DC859BD65CFA2: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:21:21,041 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - C584DC45C2D14ECD888DC859BD65CFA2: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-local-dir/output/attempt_1463534480772_0001_1_00_000000_0_10049_0/file.out
2016-05-18 01:21:21,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:21 Completed running task attempt: attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:21,044 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:21,045 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534480772_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:21,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534480772_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534481036, Process_Duration=2, Process_End_Time=1463534481038, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:21,046 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534480772_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,046 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, taskAttemptId=attempt_1463534480772_0001_1_00_000000_0, creationTime=1463534480988, allocationTime=1463534480992, startTime=1463534480995, finishTime=1463534481046, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:21,047 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534480772_0001_1_00_000000_0
2016-05-18 01:21:21,046 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:21,047 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_FINISHED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, taskId=task_1463534480772_0001_1_00_000000, startTime=1463534480995, finishTime=1463534481047, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463534480772_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481036, Process_Duration=2, Process_End_Time=1463534481038, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:21,048 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480772_0001_00_000001
2016-05-18 01:21:21,048 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:21,048 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:21,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_FINISHED]: vertexName=E56862D5128D436BAF874D56CB33CBF3, vertexId=vertex_1463534480772_0001_1_00, initRequestedTime=1463534480965, initedTime=1463534480983, startRequestedTime=1463534480967, startedTime=1463534480983, finishTime=1463534481048, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481036, Process_Duration=2, Process_End_Time=1463534481038, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534480995, firstTasksToStart=[ task_1463534480772_0001_1_00_000000 ], lastTaskFinishTime=1463534481046, lastTasksToFinish=[ task_1463534480772_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534480772_0001_1_00_000000 ], longestDurationTasks=[ task_1463534480772_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:21,050 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:21,050 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] attempt: attempt_1463534480772_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:21,050 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C584DC45C2D14ECD888DC859BD65CFA2
2016-05-18 01:21:21,050 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E56862D5128D436BAF874D56CB33CBF3, destinationVertexName=C584DC45C2D14ECD888DC859BD65CFA2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:21,051 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:21,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534480772_0001_00_000001, stoppedTime=1463534481051, exitStatus=0
2016-05-18 01:21:21,051 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:21,051 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_STARTED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, taskId=task_1463534480772_0001_1_01_000000, scheduledTime=1463534481051, launchTime=1463534481051
2016-05-18 01:21:21,052 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:21,052 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534480772_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:21,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534480772_0001_00_000002, launchTime=1463534481054
2016-05-18 01:21:21,055 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480772_0001_00_000002
2016-05-18 01:21:21,056 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480772_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21 Starting to run new task attempt: attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534480772_0001_1_01_000000_0] started. Is using containerId: [container_1463534480772_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:21,056 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,057 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,057 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:21,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0, startTime=1463534481056, containerId=container_1463534480772_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:21,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown, VertexName: C584DC45C2D14ECD888DC859BD65CFA2, VertexParallelism: 1, TaskAttemptID:attempt_1463534480772_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E56862D5128D436BAF874D56CB33CBF3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9CAC164F703F40A3A11957473A77968A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:21,058 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:21,059 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534480772_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:21,059 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480772_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:21,059 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534480772_0001_00_000001, Exiting
2016-05-18 01:21:21,059 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534480772_0001_00_000001
2016-05-18 01:21:21,059 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534480772_0001_00_000001
2016-05-18 01:21:21,059 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,065 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534480772_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:21,066 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534480772_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:21,066 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534480772_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:21,066 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534480772_0001_00_000001 completed successfully
2016-05-18 01:21:21,069 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:21,071 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:21,072 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:21,072 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:21,081 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C584DC45C2D14ECD888DC859BD65CFA2, ordinal: 1
2016-05-18 01:21:21,082 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C584DC45C2D14ECD888DC859BD65CFA2, mem on start (mb), free: 152, total: 376, max: 672
2016-05-18 01:21:21,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:21,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:21,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:21,085 INFO output.MROutput (MROutput.java:initialize(421)) - 9CAC164F703F40A3A11957473A77968A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:21,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:21,085 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[9CAC164F703F40A3A11957473A77968A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E56862D5128D436BAF874D56CB33CBF3:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:21,086 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:21,087 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:21,087 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:21,087 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[9CAC164F703F40A3A11957473A77968A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E56862D5128D436BAF874D56CB33CBF3:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:21,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:21,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E56862D5128D436BAF874D56CB33CBF3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:21,088 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E56862D5128D436BAF874D56CB33CBF3: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:21,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:21,089 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E56862D5128D436BAF874D56CB33CBF3: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:21,090 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E56862D5128D436BAF874D56CB33CBF3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:21,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E56862D5128D436BAF874D56CB33CBF3
2016-05-18 01:21:21,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:21,092 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,095 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-05-18 01:21:21,096 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E56862D5128D436BAF874D56CB33CBF3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:21,096 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: D110C49F228A441FAC07E933DF387805
2016-05-18 01:21:21,096 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 9CAC164F703F40A3A11957473A77968A
2016-05-18 01:21:21,096 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-05-18 01:21:21,096 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] D110C49F228A441FAC07E933DF387805, for 1 inputs
2016-05-18 01:21:21,098 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534480772_0001_1_00_000000_0_10049} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463534481098, TimeTaken=1, Rate=0.09 MB/s
2016-05-18 01:21:21,098 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E56862D5128D436BAF874D56CB33CBF3
2016-05-18 01:21:21,099 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,099 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E56862D5128D436BAF874D56CB33CBF3, wasInterrupted=false
2016-05-18 01:21:21,099 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,099 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - E56862D5128D436BAF874D56CB33CBF3: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:21,100 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:21,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001_wd/localmode-local-dir/output/attempt_1463534480772_0001_1_00_000000_0_10049/file.out, len=95, isLocal=true
2016-05-18 01:21:21,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:21:21,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:21,103 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:21,104 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:21:21,104 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E56862D5128D436BAF874D56CB33CBF3
2016-05-18 01:21:21,105 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E56862D5128D436BAF874D56CB33CBF3: Shuffle Runner thread complete
2016-05-18 01:21:21,104 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C584DC45C2D14ECD888DC859BD65CFA2, all 1 inputs ready in: 00:00:00.008
2016-05-18 01:21:21,106 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E56862D5128D436BAF874D56CB33CBF3: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:21,110 INFO output.MROutput (MROutput.java:close(526)) - 9CAC164F703F40A3A11957473A77968A closed
2016-05-18 01:21:21,111 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534480772_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:21,112 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,112 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C584DC45C2D14ECD888DC859BD65CFA2, mem on close (mb), free: 149, total: 376, max: 672
2016-05-18 01:21:21,112 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E56862D5128D436BAF874D56CB33CBF3
2016-05-18 01:21:21,112 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E56862D5128D436BAF874D56CB33CBF3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:21,112 INFO output.MROutput (MROutput.java:close(526)) - 9CAC164F703F40A3A11957473A77968A closed
2016-05-18 01:21:21,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:21 Completed running task attempt: attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,113 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:21,113 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534480772_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:21,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534480772_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534481105, Process_Duration=7, Process_End_Time=1463534481112, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:21:21,115 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534480772_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, taskAttemptId=attempt_1463534480772_0001_1_01_000000_0, creationTime=1463534481052, allocationTime=1463534481053, startTime=1463534481056, finishTime=1463534481115, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:21,116 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,116 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:21,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,116 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534480772_0001_00_000002
2016-05-18 01:21:21,116 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534480772_0001_1_01_000000_0
2016-05-18 01:21:21,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:TASK_FINISHED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, taskId=task_1463534480772_0001_1_01_000000, startTime=1463534481056, finishTime=1463534481117, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463534480772_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534481105, Process_Duration=7, Process_End_Time=1463534481112, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:21:21,118 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:21,118 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2]
2016-05-18 01:21:21,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:VERTEX_FINISHED]: vertexName=C584DC45C2D14ECD888DC859BD65CFA2, vertexId=vertex_1463534480772_0001_1_01, initRequestedTime=1463534480967, initedTime=1463534480972, startRequestedTime=1463534480985, startedTime=1463534480986, finishTime=1463534481118, timeTaken=132, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534481105, Process_Duration=7, Process_End_Time=1463534481112, Read_Duration=0, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463534481056, firstTasksToStart=[ task_1463534480772_0001_1_01_000000 ], lastTaskFinishTime=1463534481115, lastTasksToFinish=[ task_1463534480772_0001_1_01_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534480772_0001_1_01_000000 ], longestDurationTasks=[ task_1463534480772_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:21,120 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:21,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534480772_0001_00_000002, stoppedTime=1463534481121, exitStatus=0
2016-05-18 01:21:21,121 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:21,121 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,121 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534480772_0001_1_00 [E56862D5128D436BAF874D56CB33CBF3]
2016-05-18 01:21:21,122 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534480772_0001_1
2016-05-18 01:21:21,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534480772_0001_1
2016-05-18 01:21:21,123 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 9CAC164F703F40A3A11957473A77968A for vertex: vertex_1463534480772_0001_1_01 [C584DC45C2D14ECD888DC859BD65CFA2], outputName: 9CAC164F703F40A3A11957473A77968A
2016-05-18 01:21:21,124 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480772_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:21,125 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534480772_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:21,127 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534480772_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:21,127 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534480772_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:21,127 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534480772_0001_00_000002, Exiting
2016-05-18 01:21:21,128 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534480772_0001_00_000002
2016-05-18 01:21:21,128 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534480772_0001_00_000002
2016-05-18 01:21:21,128 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534480772_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:21,128 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534480772_0001_00_000002 completed successfully
2016-05-18 01:21:21,133 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:9CAC164F703F40A3A11957473A77968A of vertex/vertexGroup:C584DC45C2D14ECD888DC859BD65CFA2 isVertexGroupOutput:false
2016-05-18 01:21:21,133 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,135 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534480772_0001_1, queueSize=0
2016-05-18 01:21:21,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534480772_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534480772_0001_1, startTime=1463534480964, finishTime=1463534481133, timeTaken=169, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=290, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068962141, Process_Duration=9, Process_End_Time=2927068962150, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-05-18 01:21:21,136 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534480772_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:21,136 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534480772_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:21 Completed Dag: dag_1463534480772_0001_1
2016-05-18 01:21:21,136 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534480772_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:21,136 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534480772_0001_1_post
2016-05-18 01:21:21,136 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,137 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:21,137 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:21,137 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown, with id=dag_1463534480772_0001_1
2016-05-18 01:21:21,138 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A] test/(1/1) ...own/simplerelativeunknown, with id=dag_1463534480772_0001_1
2016-05-18 01:21:21,144 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1463534480772_0001
2016-05-18 01:21:21,147 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:21,147 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:21,147 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:21,147 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:21,147 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:21,148 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:21,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:21,149 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:21,149 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:21,149 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:21,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:21,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:21,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:21,152 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:21,155 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/04B79DD5BE0D4523A3976058A42059E1/C78DBD421E9C4620B452A7E910B3B02A/.tez/application_1463534480772_0001
2016-05-18 01:21:21,155 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown/_temporary
2016-05-18 01:21:21,156 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.404
2016-05-18 01:21:21,194 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-05-18 01:21:21,230 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:21,230 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:21,280 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 01:21:21,293 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-18 01:21:21,293 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:21,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 01:21:21,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:21,312 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt
2016-05-18 01:21:21,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt
2016-05-18 01:21:21,327 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-18 01:21:21,432 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting
2016-05-18 01:21:21,432 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:21,432 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:21,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-18 01:21:21,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] parallel execution of steps is enabled: false
2016-05-18 01:21:21,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] executing total steps: 1
2016-05-18 01:21:21,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] allocating management threads: 1
2016-05-18 01:21:21,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-05-18 01:21:21,434 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:21,456 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] tez session mode enabled: true
2016-05-18 01:21:21,460 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:21,461 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:21,461 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:21,469 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001 doesn't exist and is created
2016-05-18 01:21:21,490 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:21,490 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:21,491 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd
2016-05-18 01:21:21,507 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534481460_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:21,507 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:21,508 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:21,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:21,509 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:21,510 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:21,510 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:21,511 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:21,512 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:21,513 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:21,513 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:21,513 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:21,513 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:21,514 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:21,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:21,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:21,514 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:21,515 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:21,515 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:21,515 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:21,518 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:21,519 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:21,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:21,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-log-dir
2016-05-18 01:21:21,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-log-dir/history.txt.appattempt_1463534481460_0001_000000, maxErrors=10
2016-05-18 01:21:21,520 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:21,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534481460_0001_000000, appSubmitTime=1463534481506, launchTime=1463534481507
2016-05-18 01:21:21,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:21,530 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50598
2016-05-18 01:21:21,530 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534481460
2016-05-18 01:21:21,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534481460_0001_000000, startTime=1463534481532
2016-05-18 01:21:21,532 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:21,590 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:21,591 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:21,591 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463534481460_0001, dagName=[7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-18 01:21:21,609 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-18 01:21:21,609 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:21,610 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:21,610 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:21,610 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534481460_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-log-dir/dag_1463534481460_0001_1.dot
2016-05-18 01:21:21,611 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-log-dir/dag_1463534481460_0001_1-tez-dag.pb.txt
2016-05-18 01:21:21,622 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534481460_0001_1
2016-05-18 01:21:21,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,622 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-18 01:21:21 Running Dag: dag_1463534481460_0001_1
2016-05-18 01:21:21,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534481460_0001_1, submitTime=1463534481609
2016-05-18 01:21:21,638 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001/tez-conf.pb" } size: 40949 timestamp: 1463534481000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:21,638 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,639 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:21,639 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001/tez-conf.pb" } size: 40949 timestamp: 1463534481000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:21,639 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,640 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:21,640 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001/tez-conf.pb" } size: 40949 timestamp: 1463534481000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:21,641 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,641 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:21,641 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:21,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534481460_0001_1, initTime=1463534481641
2016-05-18 01:21:21,642 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534481460_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:21,642 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:21,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:DAG_STARTED]: dagID=dag_1463534481460_0001_1, startTime=1463534481642
2016-05-18 01:21:21,643 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534481460_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AAD55238C9864B52B0B5F354F665CD52 : {6AB8162D276846FF92BB98AD9417371F={InputName=6AB8162D276846FF92BB98AD9417371F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 6AB8162D276846FF92BB98AD9417371F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,643 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,644 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:21,644 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FF2A41C60CF744D79694B5D4B8567D3A : {86B1518C1AF44F2596BE434D279A17A1={InputName=86B1518C1AF44F2596BE434D279A17A1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:21,644 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 86B1518C1AF44F2596BE434D279A17A1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:21,644 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,645 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,644 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 6AB8162D276846FF92BB98AD9417371F on vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,645 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,647 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463534481460_0001, dagName=[7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-05-18 01:21:21,647 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:21,647 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 86B1518C1AF44F2596BE434D279A17A1 on vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,647 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,650 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 86B1518C1AF44F2596BE434D279A17A1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:21,652 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:21,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,652 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:21,652 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:21,653 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:21,653 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:21,653 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 86B1518C1AF44F2596BE434D279A17A1 on vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534481460_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:21,654 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B], numAdditionalOutputs=1
2016-05-18 01:21:21,654 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=647C712C0B734364A219CD35FBEA0380, vertex=vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:21,653 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 6AB8162D276846FF92BB98AD9417371F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:21,656 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:21,656 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 19 Min splitLength: 52428800 New desired splits: 1 Total length: 19 Original splits: 3
2016-05-18 01:21:21,657 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D626DFF164E54CBFB4D7697AB6EECF8B:647C712C0B734364A219CD35FBEA0380 using oldmapred API
2016-05-18 01:21:21,657 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=647C712C0B734364A219CD35FBEA0380, vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:21,657 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 19 numLocations: 1 numSplitsPerLocation: 3 numSplitsInGroup: 3 totalLength: 19 numOriginalSplits: 3 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:21,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, vertexId=vertex_1463534481460_0001_1_02, initRequestedTime=1463534481647, initedTime=1463534481654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:21,659 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:21,658 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 3
2016-05-18 01:21:21,659 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 6AB8162D276846FF92BB98AD9417371F on vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,660 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:21,661 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {86B1518C1AF44F2596BE434D279A17A1=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:21,661 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:21,662 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {6AB8162D276846FF92BB98AD9417371F=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:21,662 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] parallelism set to 1
2016-05-18 01:21:21,663 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:21,662 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] parallelism set to 1
2016-05-18 01:21:21,663 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:21,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, vertexId=vertex_1463534481460_0001_1_01, initRequestedTime=1463534481643, initedTime=1463534481664, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:21,664 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] with distanceFromRoot: 0
2016-05-18 01:21:21,664 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:21,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, vertexId=vertex_1463534481460_0001_1_00, initRequestedTime=1463534481644, initedTime=1463534481664, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:21,665 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] with distanceFromRoot: 0
2016-05-18 01:21:21,665 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:21,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534481460_0001_1_01, startRequestedTime=1463534481647, startedTime=1463534481665
2016-05-18 01:21:21,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534481460_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:21,665 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:21,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534481460_0001_1_00, startRequestedTime=1463534481647, startedTime=1463534481666
2016-05-18 01:21:21,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534481460_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:21,666 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:21,666 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534481460_0001_1_01 for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] numStartedSources: 1 numSources: 2
2016-05-18 01:21:21,666 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] numStartedSources: 1 numSources: 2
2016-05-18 01:21:21,666 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534481460_0001_1_00 for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] numStartedSources: 2 numSources: 2
2016-05-18 01:21:21,667 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] with distanceFromRoot: 1
2016-05-18 01:21:21,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534481460_0001_1_02, startRequestedTime=1463534481666, startedTime=1463534481667
2016-05-18 01:21:21,667 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:21,668 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FF2A41C60CF744D79694B5D4B8567D3A
2016-05-18 01:21:21,674 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D626DFF164E54CBFB4D7697AB6EECF8B with 0 source tasks and 1 pending tasks
2016-05-18 01:21:21,674 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,674 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AAD55238C9864B52B0B5F354F665CD52
2016-05-18 01:21:21,675 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,675 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FF2A41C60CF744D79694B5D4B8567D3A in vertex: D626DFF164E54CBFB4D7697AB6EECF8B numBipartiteSourceTasks: 1
2016-05-18 01:21:21,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_STARTED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, taskId=task_1463534481460_0001_1_00_000000, scheduledTime=1463534481675, launchTime=1463534481675
2016-05-18 01:21:21,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_STARTED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, taskId=task_1463534481460_0001_1_01_000000, scheduledTime=1463534481676, launchTime=1463534481676
2016-05-18 01:21:21,677 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,677 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,677 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: AAD55238C9864B52B0B5F354F665CD52 in vertex: D626DFF164E54CBFB4D7697AB6EECF8B numBipartiteSourceTasks: 2
2016-05-18 01:21:21,678 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:21,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534481460_0001_00_000001, launchTime=1463534481680
2016-05-18 01:21:21,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534481460_0001_00_000002, launchTime=1463534481680
2016-05-18 01:21:21,681 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000001
2016-05-18 01:21:21,682 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000002
2016-05-18 01:21:21,683 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534481460_0001_1_00_000000_0] started. Is using containerId: [container_1463534481460_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:21,683 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0, startTime=1463534481683, containerId=container_1463534481460_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:21 Starting to run new task attempt: attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,685 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,685 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,686 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:21,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: FF2A41C60CF744D79694B5D4B8567D3A, VertexParallelism: 1, TaskAttemptID:attempt_1463534481460_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86B1518C1AF44F2596BE434D279A17A1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D626DFF164E54CBFB4D7697AB6EECF8B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:21,686 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:21,687 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] submitted tez dag to app master: application_1463534481460_0001, with dag id: dag_1463534481460_0001_1
2016-05-18 01:21:21,695 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:21,696 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534481460_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:21,696 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534481460_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:21,697 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534481460_0001_1_01_000000_0] started. Is using containerId: [container_1463534481460_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:21,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0, startTime=1463534481697, containerId=container_1463534481460_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:21,698 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21 Starting to run new task attempt: attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,699 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,700 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,700 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:21,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: AAD55238C9864B52B0B5F354F665CD52, VertexParallelism: 1, TaskAttemptID:attempt_1463534481460_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6AB8162D276846FF92BB98AD9417371F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D626DFF164E54CBFB4D7697AB6EECF8B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:21,701 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:21,701 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,702 INFO input.MRInput (MRInput.java:initialize(440)) - 86B1518C1AF44F2596BE434D279A17A1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:21,702 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 86B1518C1AF44F2596BE434D279A17A1
2016-05-18 01:21:21,706 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534481460_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 01:21:21,706 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534481460_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:21,710 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:21,710 INFO input.MRInput (MRInput.java:initialize(440)) - 6AB8162D276846FF92BB98AD9417371F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:21,710 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 6AB8162D276846FF92BB98AD9417371F
2016-05-18 01:21:21,711 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF2A41C60CF744D79694B5D4B8567D3A, ordinal: 0
2016-05-18 01:21:21,712 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A41C60CF744D79694B5D4B8567D3A, mem on start (mb), free: 232, total: 375, max: 672
2016-05-18 01:21:21,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:21,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:21,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:21,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:21,713 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[86B1518C1AF44F2596BE434D279A17A1:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D626DFF164E54CBFB4D7697AB6EECF8B:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:21,713 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:21,713 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:21,714 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:21,714 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[86B1518C1AF44F2596BE434D279A17A1:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D626DFF164E54CBFB4D7697AB6EECF8B:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:21,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:21,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 86B1518C1AF44F2596BE434D279A17A1
2016-05-18 01:21:21,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 86B1518C1AF44F2596BE434D279A17A1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:21,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:21,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:21,716 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,720 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:21,720 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"] streamed: true, id: 86B1518C1AF44F2596BE434D279A17A1
2016-05-18 01:21:21,720 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:21,722 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 86B1518C1AF44F2596BE434D279A17A1 initialized RecordReader from event
2016-05-18 01:21:21,722 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AAD55238C9864B52B0B5F354F665CD52, ordinal: 1
2016-05-18 01:21:21,722 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD55238C9864B52B0B5F354F665CD52, mem on start (mb), free: 230, total: 375, max: 672
2016-05-18 01:21:21,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:21,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:21,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:21,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:21,722 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6AB8162D276846FF92BB98AD9417371F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D626DFF164E54CBFB4D7697AB6EECF8B:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:21,721 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 2B81758E390445EEBF40D50C742F0CA8
2016-05-18 01:21:21,723 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 2B81758E390445EEBF40D50C742F0CA8
2016-05-18 01:21:21,724 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:21,724 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:21,725 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:21,725 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6AB8162D276846FF92BB98AD9417371F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D626DFF164E54CBFB4D7697AB6EECF8B:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:21,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:21,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 6AB8162D276846FF92BB98AD9417371F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:21,727 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D626DFF164E54CBFB4D7697AB6EECF8B using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@1fea4691, comparator=cascading.tuple.hadoop.util.TupleComparator@4dbee9f5, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:21,728 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D626DFF164E54CBFB4D7697AB6EECF8B: , UsingHashComparator=false
2016-05-18 01:21:21,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 6AB8162D276846FF92BB98AD9417371F
2016-05-18 01:21:21,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:21,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:21,742 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,746 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:0+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:9+9, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt:18+1], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=19}
2016-05-18 01:21:21,746 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 6AB8162D276846FF92BB98AD9417371F initialized RecordReader from event
2016-05-18 01:21:21,753 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:21,754 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:21,754 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D626DFF164E54CBFB4D7697AB6EECF8B: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:21,754 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D626DFF164E54CBFB4D7697AB6EECF8B: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:21,754 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:21,754 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/lower.txt"]
2016-05-18 01:21:21,755 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF2A41C60CF744D79694B5D4B8567D3A, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:21,755 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"] streamed: true, id: 6AB8162D276846FF92BB98AD9417371F
2016-05-18 01:21:21,755 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 2B81758E390445EEBF40D50C742F0CA8
2016-05-18 01:21:21,756 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 2B81758E390445EEBF40D50C742F0CA8
2016-05-18 01:21:21,758 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,758 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF2A41C60CF744D79694B5D4B8567D3A, mem on close (mb), free: 238, total: 376, max: 672
2016-05-18 01:21:21,758 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D626DFF164E54CBFB4D7697AB6EECF8B: Starting flush of map output
2016-05-18 01:21:21,759 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D626DFF164E54CBFB4D7697AB6EECF8B: Span0.length = 5, perItem = 15
2016-05-18 01:21:21,759 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D626DFF164E54CBFB4D7697AB6EECF8B: done sorting span=0, length=5, time=0
2016-05-18 01:21:21,759 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D626DFF164E54CBFB4D7697AB6EECF8B: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:21:21,761 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D626DFF164E54CBFB4D7697AB6EECF8B: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-local-dir/output/attempt_1463534481460_0001_1_00_000000_0_10055_0/file.out
2016-05-18 01:21:21,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:21 Completed running task attempt: attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,768 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:21,768 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534481460_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:21,769 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D626DFF164E54CBFB4D7697AB6EECF8B using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@762e0806, comparator=cascading.tuple.hadoop.util.TupleComparator@4810fe87, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:21,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534481460_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534481755, Process_Duration=3, Process_End_Time=1463534481758, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:21,769 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D626DFF164E54CBFB4D7697AB6EECF8B: , UsingHashComparator=false
2016-05-18 01:21:21,782 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:21,783 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:21,783 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D626DFF164E54CBFB4D7697AB6EECF8B: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:21,784 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D626DFF164E54CBFB4D7697AB6EECF8B: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:21,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, taskAttemptId=attempt_1463534481460_0001_1_00_000000_0, creationTime=1463534481676, allocationTime=1463534481679, startTime=1463534481683, finishTime=1463534481783, timeTaken=100, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:21,785 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534481460_0001_1_00_000000_0
2016-05-18 01:21:21,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_FINISHED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, taskId=task_1463534481460_0001_1_00_000000, startTime=1463534481683, finishTime=1463534481785, timeTaken=102, status=SUCCEEDED, successfulAttemptID=attempt_1463534481460_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481755, Process_Duration=3, Process_End_Time=1463534481758, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:21,786 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:21,786 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,783 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534481460_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,788 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,788 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:21,788 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,789 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000001
2016-05-18 01:21:21,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_FINISHED]: vertexName=FF2A41C60CF744D79694B5D4B8567D3A, vertexId=vertex_1463534481460_0001_1_00, initRequestedTime=1463534481644, initedTime=1463534481664, startRequestedTime=1463534481647, startedTime=1463534481666, finishTime=1463534481788, timeTaken=122, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481755, Process_Duration=3, Process_End_Time=1463534481758, Read_Duration=2, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534481683, firstTasksToStart=[ task_1463534481460_0001_1_00_000000 ], lastTaskFinishTime=1463534481783, lastTasksToFinish=[ task_1463534481460_0001_1_00_000000 ], minTaskDuration=100, maxTaskDuration=100, avgTaskDuration=100.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534481460_0001_1_00_000000 ], longestDurationTasks=[ task_1463534481460_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:21,792 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:21,792 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] attempt: attempt_1463534481460_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:21,792 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/upper.txt"]
2016-05-18 01:21:21,793 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AAD55238C9864B52B0B5F354F665CD52, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:21,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534481460_0001_00_000001, stoppedTime=1463534481793, exitStatus=0
2016-05-18 01:21:21,794 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:21,794 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,796 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,796 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AAD55238C9864B52B0B5F354F665CD52, mem on close (mb), free: 137, total: 376, max: 672
2016-05-18 01:21:21,796 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534481460_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:21,797 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D626DFF164E54CBFB4D7697AB6EECF8B: Starting flush of map output
2016-05-18 01:21:21,798 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D626DFF164E54CBFB4D7697AB6EECF8B: Span0.length = 5, perItem = 15
2016-05-18 01:21:21,798 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D626DFF164E54CBFB4D7697AB6EECF8B: done sorting span=0, length=5, time=0
2016-05-18 01:21:21,798 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D626DFF164E54CBFB4D7697AB6EECF8B: Heap = SpanIterator<0:4> (span=Span[16777216,75]),
2016-05-18 01:21:21,799 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D626DFF164E54CBFB4D7697AB6EECF8B: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-local-dir/output/attempt_1463534481460_0001_1_01_000000_0_10058_0/file.out
2016-05-18 01:21:21,801 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534481460_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:21,802 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:21,802 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534481460_0001_00_000001, Exiting
2016-05-18 01:21:21,802 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534481460_0001_00_000001
2016-05-18 01:21:21,802 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534481460_0001_00_000001
2016-05-18 01:21:21,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:21 Completed running task attempt: attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,804 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534481460_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:21,806 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534481460_0001_00_000001 completed successfully
2016-05-18 01:21:21,805 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:21,807 WARN task.TaskReporter (TaskReporter.java:call(214)) - Exiting TaskReporter thread with pending queue size=2
2016-05-18 01:21:21,807 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534481460_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:21,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534481460_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534481793, Process_Duration=3, Process_End_Time=1463534481796, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-05-18 01:21:21,810 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534481460_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,810 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,810 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:21,810 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, taskAttemptId=attempt_1463534481460_0001_1_01_000000_0, creationTime=1463534481677, allocationTime=1463534481680, startTime=1463534481697, finishTime=1463534481810, timeTaken=113, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:21,811 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534481460_0001_1_01_000000_0
2016-05-18 01:21:21,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_FINISHED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, taskId=task_1463534481460_0001_1_01_000000, startTime=1463534481697, finishTime=1463534481811, timeTaken=114, status=SUCCEEDED, successfulAttemptID=attempt_1463534481460_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481793, Process_Duration=3, Process_End_Time=1463534481796, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-05-18 01:21:21,812 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:21,812 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,812 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000002
2016-05-18 01:21:21,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_FINISHED]: vertexName=AAD55238C9864B52B0B5F354F665CD52, vertexId=vertex_1463534481460_0001_1_01, initRequestedTime=1463534481643, initedTime=1463534481664, startRequestedTime=1463534481647, startedTime=1463534481665, finishTime=1463534481812, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534481793, Process_Duration=3, Process_End_Time=1463534481796, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1463534481697, firstTasksToStart=[ task_1463534481460_0001_1_01_000000 ], lastTaskFinishTime=1463534481810, lastTasksToFinish=[ task_1463534481460_0001_1_01_000000 ], minTaskDuration=113, maxTaskDuration=113, avgTaskDuration=113.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534481460_0001_1_01_000000 ], longestDurationTasks=[ task_1463534481460_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:21,813 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:21,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534481460_0001_00_000002, stoppedTime=1463534481813, exitStatus=0
2016-05-18 01:21:21,814 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] attempt: attempt_1463534481460_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:21,814 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:21,814 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,814 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D626DFF164E54CBFB4D7697AB6EECF8B
2016-05-18 01:21:21,814 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FF2A41C60CF744D79694B5D4B8567D3A, destinationVertexName=D626DFF164E54CBFB4D7697AB6EECF8B. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:21,814 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=AAD55238C9864B52B0B5F354F665CD52, destinationVertexName=D626DFF164E54CBFB4D7697AB6EECF8B. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:21,814 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,815 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534481460_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:21,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_STARTED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, taskId=task_1463534481460_0001_1_02_000000, scheduledTime=1463534481815, launchTime=1463534481815
2016-05-18 01:21:21,815 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534481460_0001_00_000003, launchTime=1463534481817
2016-05-18 01:21:21,818 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000003
2016-05-18 01:21:21,819 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534481460_0001_1_02_000000_0] started. Is using containerId: [container_1463534481460_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 01:21:21,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0, startTime=1463534481818, containerId=container_1463534481460_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 01:21:21,820 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21 Starting to run new task attempt: attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,820 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,821 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,821 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:21,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D626DFF164E54CBFB4D7697AB6EECF8B, VertexParallelism: 1, TaskAttemptID:attempt_1463534481460_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF2A41C60CF744D79694B5D4B8567D3A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=AAD55238C9864B52B0B5F354F665CD52, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=647C712C0B734364A219CD35FBEA0380, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:21,822 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:21,823 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,824 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534481460_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:21,824 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:21,825 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534481460_0001_00_000002, Exiting
2016-05-18 01:21:21,825 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534481460_0001_00_000002
2016-05-18 01:21:21,825 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534481460_0001_00_000002
2016-05-18 01:21:21,825 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534481460_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,825 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534481460_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 01:21:21,827 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:21,829 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:21,830 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534481460_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:21,831 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534481460_0001_00_000002 completed successfully
2016-05-18 01:21:21,832 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:21,838 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D626DFF164E54CBFB4D7697AB6EECF8B, ordinal: 2
2016-05-18 01:21:21,838 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D626DFF164E54CBFB4D7697AB6EECF8B, mem on start (mb), free: 129, total: 376, max: 672
2016-05-18 01:21:21,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:21,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 01:21:21,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:21,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:21,842 INFO output.MROutput (MROutput.java:initialize(421)) - 647C712C0B734364A219CD35FBEA0380: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:21,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:21,842 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[647C712C0B734364A219CD35FBEA0380:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [FF2A41C60CF744D79694B5D4B8567D3A:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [AAD55238C9864B52B0B5F354F665CD52:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:21,843 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:21,843 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 01:21:21,843 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 01:21:21,843 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[647C712C0B734364A219CD35FBEA0380:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [FF2A41C60CF744D79694B5D4B8567D3A:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [AAD55238C9864B52B0B5F354F665CD52:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:21,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:21,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FF2A41C60CF744D79694B5D4B8567D3A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: AAD55238C9864B52B0B5F354F665CD52 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:21,844 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FF2A41C60CF744D79694B5D4B8567D3A: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:21,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 01:21:21,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 01:21:21,844 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FF2A41C60CF744D79694B5D4B8567D3A: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:21,846 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - AAD55238C9864B52B0B5F354F665CD52: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:21,846 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - AAD55238C9864B52B0B5F354F665CD52: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:21,847 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FF2A41C60CF744D79694B5D4B8567D3A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:21,847 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: AAD55238C9864B52B0B5F354F665CD52 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:21,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FF2A41C60CF744D79694B5D4B8567D3A
2016-05-18 01:21:21,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:21,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: AAD55238C9864B52B0B5F354F665CD52
2016-05-18 01:21:21,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:21,852 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,853 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A41C60CF744D79694B5D4B8567D3A: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:21,853 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AAD55238C9864B52B0B5F354F665CD52: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:21,873 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534481460_0001_1_00_000000_0_10055} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463534481873, TimeTaken=3, Rate=0.03 MB/s
2016-05-18 01:21:21,874 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FF2A41C60CF744D79694B5D4B8567D3A
2016-05-18 01:21:21,873 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-05-18 01:21:21,874 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,874 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FF2A41C60CF744D79694B5D4B8567D3A, wasInterrupted=false
2016-05-18 01:21:21,875 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: 2B81758E390445EEBF40D50C742F0CA8
2016-05-18 01:21:21,876 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,876 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: 647C712C0B734364A219CD35FBEA0380
2016-05-18 01:21:21,876 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - FF2A41C60CF744D79694B5D4B8567D3A: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:21,877 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-05-18 01:21:21,877 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:21,877 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 2B81758E390445EEBF40D50C742F0CA8, for 2 inputs
2016-05-18 01:21:21,877 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 2B81758E390445EEBF40D50C742F0CA8, for 2 inputs
2016-05-18 01:21:21,878 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-local-dir/output/attempt_1463534481460_0001_1_00_000000_0_10055/file.out, len=95, isLocal=true
2016-05-18 01:21:21,879 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:21:21,879 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:21,879 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:21:21,879 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:21,879 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 01:21:21,879 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:21:21,880 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FF2A41C60CF744D79694B5D4B8567D3A
2016-05-18 01:21:21,880 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FF2A41C60CF744D79694B5D4B8567D3A: Shuffle Runner thread complete
2016-05-18 01:21:21,881 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534481460_0001_1_01_000000_0_10058} to DISK_DIRECT, csize=95, dsize=91, EndTime=1463534481881, TimeTaken=9, Rate=0.01 MB/s
2016-05-18 01:21:21,881 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : AAD55238C9864B52B0B5F354F665CD52
2016-05-18 01:21:21,881 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,881 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: AAD55238C9864B52B0B5F354F665CD52, wasInterrupted=false
2016-05-18 01:21:21,884 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:21,885 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:21,886 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001_wd/localmode-local-dir/output/attempt_1463534481460_0001_1_01_000000_0_10058/file.out, len=95, isLocal=true
2016-05-18 01:21:21,886 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 95 bytes from disk
2016-05-18 01:21:21,886 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:21,886 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:21,887 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-05-18 01:21:21,887 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : AAD55238C9864B52B0B5F354F665CD52
2016-05-18 01:21:21,887 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - AAD55238C9864B52B0B5F354F665CD52: Shuffle Runner thread complete
2016-05-18 01:21:21,887 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D626DFF164E54CBFB4D7697AB6EECF8B, all 2 inputs ready in: 00:00:00.008
2016-05-18 01:21:21,888 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FF2A41C60CF744D79694B5D4B8567D3A: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:21,891 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - AAD55238C9864B52B0B5F354F665CD52: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:21,896 INFO output.MROutput (MROutput.java:close(526)) - 647C712C0B734364A219CD35FBEA0380 closed
2016-05-18 01:21:21,898 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534481460_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 01:21:21,899 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,900 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D626DFF164E54CBFB4D7697AB6EECF8B, mem on close (mb), free: 340, total: 376, max: 672
2016-05-18 01:21:21,900 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FF2A41C60CF744D79694B5D4B8567D3A
2016-05-18 01:21:21,900 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FF2A41C60CF744D79694B5D4B8567D3A: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:21,900 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: AAD55238C9864B52B0B5F354F665CD52
2016-05-18 01:21:21,901 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AAD55238C9864B52B0B5F354F665CD52: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:21,901 INFO output.MROutput (MROutput.java:close(526)) - 647C712C0B734364A219CD35FBEA0380 closed
2016-05-18 01:21:21,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:21 Completed running task attempt: attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,902 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0, askedToStop=false
2016-05-18 01:21:21,902 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534481460_0001_1_02_000000_0, stopRequested=false
2016-05-18 01:21:21,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534481460_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=66, MERGE_PHASE_TIME=77, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534481888, Process_Duration=11, Process_End_Time=1463534481899, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-05-18 01:21:21,905 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534481460_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,905 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:21,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, taskAttemptId=attempt_1463534481460_0001_1_02_000000_0, creationTime=1463534481815, allocationTime=1463534481816, startTime=1463534481818, finishTime=1463534481905, timeTaken=87, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:21,905 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534481460_0001_1_02_000000_0
2016-05-18 01:21:21,905 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:21,906 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:TASK_FINISHED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, taskId=task_1463534481460_0001_1_02_000000, startTime=1463534481818, finishTime=1463534481905, timeTaken=87, status=SUCCEEDED, successfulAttemptID=attempt_1463534481460_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=66, MERGE_PHASE_TIME=77, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534481888, Process_Duration=11, Process_End_Time=1463534481899, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-05-18 01:21:21,906 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534481460_0001_00_000003
2016-05-18 01:21:21,906 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:21,906 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B]
2016-05-18 01:21:21,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:VERTEX_FINISHED]: vertexName=D626DFF164E54CBFB4D7697AB6EECF8B, vertexId=vertex_1463534481460_0001_1_02, initRequestedTime=1463534481647, initedTime=1463534481654, startRequestedTime=1463534481666, startedTime=1463534481667, finishTime=1463534481906, timeTaken=239, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=66, MERGE_PHASE_TIME=77, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534481888, Process_Duration=11, Process_End_Time=1463534481899, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1463534481818, firstTasksToStart=[ task_1463534481460_0001_1_02_000000 ], lastTaskFinishTime=1463534481905, lastTasksToFinish=[ task_1463534481460_0001_1_02_000000 ], minTaskDuration=87, maxTaskDuration=87, avgTaskDuration=87.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534481460_0001_1_02_000000 ], longestDurationTasks=[ task_1463534481460_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:21,907 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:21,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534481460_0001_00_000003, stoppedTime=1463534481908, exitStatus=0
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534481460_0001_1_01 [AAD55238C9864B52B0B5F354F665CD52]
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534481460_0001_1_00 [FF2A41C60CF744D79694B5D4B8567D3A]
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534481460_0001_1
2016-05-18 01:21:21,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534481460_0001_1
2016-05-18 01:21:21,908 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 647C712C0B734364A219CD35FBEA0380 for vertex: vertex_1463534481460_0001_1_02 [D626DFF164E54CBFB4D7697AB6EECF8B], outputName: 647C712C0B734364A219CD35FBEA0380
2016-05-18 01:21:21,909 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534481460_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:21,910 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534481460_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 01:21:21,917 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534481460_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 01:21:21,917 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534481460_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:21,917 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534481460_0001_00_000003, Exiting
2016-05-18 01:21:21,917 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534481460_0001_00_000003
2016-05-18 01:21:21,917 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534481460_0001_00_000003
2016-05-18 01:21:21,917 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534481460_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:21,917 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534481460_0001_00_000003 completed successfully
2016-05-18 01:21:21,918 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:647C712C0B734364A219CD35FBEA0380 of vertex/vertexGroup:D626DFF164E54CBFB4D7697AB6EECF8B isVertexGroupOutput:false
2016-05-18 01:21:21,918 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 01:21:21,920 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534481460_0001_1, queueSize=0
2016-05-18 01:21:21,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534481460_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534481460_0001_1, startTime=1463534481642, finishTime=1463534481918, timeTaken=276, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=620, AM_GC_TIME_MILLIS=20, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=10, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=66, MERGE_PHASE_TIME=77, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390603445436, Process_Duration=17, Process_End_Time=4390603445453, Read_Duration=4, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-05-18 01:21:21,921 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534481460_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:21,921 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534481460_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:21 Completed Dag: dag_1463534481460_0001_1
2016-05-18 01:21:21,921 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534481460_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:21,922 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534481460_0001_1_post
2016-05-18 01:21:21,922 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:21,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:21,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:21,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup, with id=dag_1463534481460_0001_1
2016-05-18 01:21:21,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395] lower*upper/(1/1) ...estCoGroup/complexcogroup, with id=dag_1463534481460_0001_1
2016-05-18 01:21:21,930 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1463534481460_0001
2016-05-18 01:21:21,932 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:21,932 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:21,932 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:21,932 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:21,933 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:21,933 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:21,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:21,934 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:21,934 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:21,934 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:21,934 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:21,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:21,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:21,935 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:21,939 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7277E383F4C34F9DB9816167E9507201/E5BC15B64F4A4B6484F4B0EDD7AA6395/.tez/application_1463534481460_0001
2016-05-18 01:21:21,940 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup/_temporary
2016-05-18 01:21:21,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [lower*upper] completed in: 00:00.507
2016-05-18 01:21:22,016 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-05-18 01:21:22,044 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:22,045 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:22,084 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:21:22,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 01:21:22,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:22,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:22,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:22,110 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:22,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:22,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:22,126 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-05-18 01:21:22,126 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:22,144 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:22,149 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:22,150 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:22,150 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:22,163 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001 doesn't exist and is created
2016-05-18 01:21:22,186 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:22,186 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:22,187 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001_wd
2016-05-18 01:21:22,204 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534482149_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:22,204 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:22,205 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:22,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:22,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:22,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:22,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:22,207 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:22,208 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:22,208 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:22,208 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:22,209 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:22,209 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:22,210 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:22,210 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:22,211 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:22,212 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:22,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:22,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001_wd/localmode-log-dir
2016-05-18 01:21:22,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001_wd/localmode-log-dir/history.txt.appattempt_1463534482149_0001_000000, maxErrors=10
2016-05-18 01:21:22,213 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:22,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534482149_0001_000000, appSubmitTime=1463534482204, launchTime=1463534482204
2016-05-18 01:21:22,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:22,224 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50170
2016-05-18 01:21:22,224 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534482149
2016-05-18 01:21:22,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534482149_0001_000000, startTime=1463534482226
2016-05-18 01:21:22,227 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:22,286 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:22,286 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:22,286 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463534482149_0001, dagName=[B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace
2016-05-18 01:21:22,305 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace
2016-05-18 01:21:22,305 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:22,306 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534482149_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001_wd/localmode-log-dir/dag_1463534482149_0001_1.dot
2016-05-18 01:21:22,307 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001_wd/localmode-log-dir/dag_1463534482149_0001_1-tez-dag.pb.txt
2016-05-18 01:21:22,311 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534482149_0001_1
2016-05-18 01:21:22,311 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,311 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace
2016-05-18 01:21:22 Running Dag: dag_1463534482149_0001_1
2016-05-18 01:21:22,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534482149_0001_1, submitTime=1463534482305
2016-05-18 01:21:22,326 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001/tez-conf.pb" } size: 40949 timestamp: 1463534482000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:22,327 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,327 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,327 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:22,328 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:22,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534482149_0001_1, initTime=1463534482328
2016-05-18 01:21:22,329 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482149_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:22,329 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:22,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:DAG_STARTED]: dagID=dag_1463534482149_0001_1, startTime=1463534482330
2016-05-18 01:21:22,330 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482149_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:22,330 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E7806CD32305468BABB5FDCB211B11C4 : {0691B038A6F24EE8BE2589B1E0EF25EB={InputName=0691B038A6F24EE8BE2589B1E0EF25EB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:22,330 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463534482149_0001, dagName=[B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace
2016-05-18 01:21:22,330 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0691B038A6F24EE8BE2589B1E0EF25EB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:22,331 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,331 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,331 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,331 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,332 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:22,332 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0691B038A6F24EE8BE2589B1E0EF25EB on vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,334 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0691B038A6F24EE8BE2589B1E0EF25EB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:22,339 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:22,339 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:22,339 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:22,340 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:22,340 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0691B038A6F24EE8BE2589B1E0EF25EB on vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,344 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:22,344 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0691B038A6F24EE8BE2589B1E0EF25EB=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:22,344 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] parallelism set to 1
2016-05-18 01:21:22,345 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:22,345 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4], numAdditionalOutputs=1
2016-05-18 01:21:22,345 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=85CB1C1E52CA4B28B8EA7BD017854270, vertex=vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:22,347 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E7806CD32305468BABB5FDCB211B11C4:85CB1C1E52CA4B28B8EA7BD017854270 using oldmapred API
2016-05-18 01:21:22,348 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=85CB1C1E52CA4B28B8EA7BD017854270, vertexName=E7806CD32305468BABB5FDCB211B11C4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:22,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, vertexId=vertex_1463534482149_0001_1_00, initRequestedTime=1463534482330, initedTime=1463534482345, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:22,349 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] with distanceFromRoot: 0
2016-05-18 01:21:22,349 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:22,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534482149_0001_1_00, startRequestedTime=1463534482332, startedTime=1463534482349
2016-05-18 01:21:22,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534482149_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:22,350 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:22,350 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E7806CD32305468BABB5FDCB211B11C4
2016-05-18 01:21:22,350 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:TASK_STARTED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, taskId=task_1463534482149_0001_1_00_000000, scheduledTime=1463534482351, launchTime=1463534482351
2016-05-18 01:21:22,352 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,353 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:22,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534482149_0001_00_000001, launchTime=1463534482354
2016-05-18 01:21:22,355 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534482149_0001, with dag id: dag_1463534482149_0001_1
2016-05-18 01:21:22,355 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482149_0001_00_000001
2016-05-18 01:21:22,358 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482149_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22 Starting to run new task attempt: attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,359 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,359 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,359 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:22,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534482149_0001_1_00_000000_0] started. Is using containerId: [container_1463534482149_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:22,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace, VertexName: E7806CD32305468BABB5FDCB211B11C4, VertexParallelism: 1, TaskAttemptID:attempt_1463534482149_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0691B038A6F24EE8BE2589B1E0EF25EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=85CB1C1E52CA4B28B8EA7BD017854270, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:22,360 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:22,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0, startTime=1463534482357, containerId=container_1463534482149_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:22,361 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534482149_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:22,361 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534482149_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:22,361 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,366 INFO input.MRInput (MRInput.java:initialize(440)) - 0691B038A6F24EE8BE2589B1E0EF25EB using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:22,366 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0691B038A6F24EE8BE2589B1E0EF25EB
2016-05-18 01:21:22,367 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:22,373 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E7806CD32305468BABB5FDCB211B11C4, ordinal: 0
2016-05-18 01:21:22,373 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7806CD32305468BABB5FDCB211B11C4, mem on start (mb), free: 252, total: 376, max: 672
2016-05-18 01:21:22,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:22,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:22,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:22,377 INFO output.MROutput (MROutput.java:initialize(421)) - 85CB1C1E52CA4B28B8EA7BD017854270: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:22,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:22,377 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0691B038A6F24EE8BE2589B1E0EF25EB:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [85CB1C1E52CA4B28B8EA7BD017854270:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:22,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:22,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:22,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:21:22,378 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0691B038A6F24EE8BE2589B1E0EF25EB:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [85CB1C1E52CA4B28B8EA7BD017854270:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:22,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:22,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0691B038A6F24EE8BE2589B1E0EF25EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:22,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0691B038A6F24EE8BE2589B1E0EF25EB
2016-05-18 01:21:22,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:22,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:22,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:22,380 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,381 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:22,382 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0691B038A6F24EE8BE2589B1E0EF25EB initialized RecordReader from event
2016-05-18 01:21:22,384 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:22,385 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 0691B038A6F24EE8BE2589B1E0EF25EB
2016-05-18 01:21:22,385 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 85CB1C1E52CA4B28B8EA7BD017854270
2016-05-18 01:21:22,385 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-05-18 01:21:22,385 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:22,386 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E7806CD32305468BABB5FDCB211B11C4, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:22,388 INFO output.MROutput (MROutput.java:close(526)) - 85CB1C1E52CA4B28B8EA7BD017854270 closed
2016-05-18 01:21:22,389 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534482149_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:21:22,390 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,390 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7806CD32305468BABB5FDCB211B11C4, mem on close (mb), free: 250, total: 376, max: 672
2016-05-18 01:21:22,391 INFO output.MROutput (MROutput.java:close(526)) - 85CB1C1E52CA4B28B8EA7BD017854270 closed
2016-05-18 01:21:22,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:22 Completed running task attempt: attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,393 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:22,393 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534482149_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:22,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534482149_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1463534482386, Process_Duration=4, Process_End_Time=1463534482390, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-05-18 01:21:22,395 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534482149_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:22,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, taskAttemptId=attempt_1463534482149_0001_1_00_000000_0, creationTime=1463534482352, allocationTime=1463534482354, startTime=1463534482357, finishTime=1463534482395, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:22,395 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:22,396 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534482149_0001_1_00_000000_0
2016-05-18 01:21:22,396 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:22,396 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,396 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482149_0001_00_000001
2016-05-18 01:21:22,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:TASK_FINISHED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, taskId=task_1463534482149_0001_1_00_000000, startTime=1463534482357, finishTime=1463534482396, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463534482149_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534482386, Process_Duration=4, Process_End_Time=1463534482390, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:21:22,400 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:22,401 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4]
2016-05-18 01:21:22,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:VERTEX_FINISHED]: vertexName=E7806CD32305468BABB5FDCB211B11C4, vertexId=vertex_1463534482149_0001_1_00, initRequestedTime=1463534482330, initedTime=1463534482345, startRequestedTime=1463534482332, startedTime=1463534482349, finishTime=1463534482401, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534482386, Process_Duration=4, Process_End_Time=1463534482390, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1463534482357, firstTasksToStart=[ task_1463534482149_0001_1_00_000000 ], lastTaskFinishTime=1463534482395, lastTasksToFinish=[ task_1463534482149_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534482149_0001_1_00_000000 ], longestDurationTasks=[ task_1463534482149_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:22,402 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:22,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534482149_0001_00_000001, stoppedTime=1463534482402, exitStatus=0
2016-05-18 01:21:22,402 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:21:22,402 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:22,403 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534482149_0001_1
2016-05-18 01:21:22,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534482149_0001_1
2016-05-18 01:21:22,403 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482149_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:22,403 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534482149_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:22,403 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 85CB1C1E52CA4B28B8EA7BD017854270 for vertex: vertex_1463534482149_0001_1_00 [E7806CD32305468BABB5FDCB211B11C4], outputName: 85CB1C1E52CA4B28B8EA7BD017854270
2016-05-18 01:21:22,410 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534482149_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:22,411 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482149_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:22,411 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534482149_0001_00_000001, Exiting
2016-05-18 01:21:22,411 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534482149_0001_00_000001
2016-05-18 01:21:22,411 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534482149_0001_00_000001
2016-05-18 01:21:22,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534482149_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:22,415 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534482149_0001_00_000001 completed successfully
2016-05-18 01:21:22,419 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:85CB1C1E52CA4B28B8EA7BD017854270 of vertex/vertexGroup:E7806CD32305468BABB5FDCB211B11C4 isVertexGroupOutput:false
2016-05-18 01:21:22,419 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:22,420 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534482149_0001_1, queueSize=0
2016-05-18 01:21:22,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482149_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534482149_0001_1, startTime=1463534482329, finishTime=1463534482419, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=170, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1463534482386, Process_Duration=4, Process_End_Time=1463534482390, Read_Duration=0, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-05-18 01:21:22,420 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534482149_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:22,420 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482149_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:22 Completed Dag: dag_1463534482149_0001_1
2016-05-18 01:21:22,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534482149_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:22,421 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534482149_0001_1_post
2016-05-18 01:21:22,421 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:22,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:22,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace, with id=dag_1463534482149_0001_1
2016-05-18 01:21:22,421 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6] test/(1/1) ...tform/testReplace/replace, with id=dag_1463534482149_0001_1
2016-05-18 01:21:22,430 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1463534482149_0001
2016-05-18 01:21:22,436 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:22,437 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:22,437 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:22,437 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:22,437 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:22,437 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:22,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:22,438 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:22,438 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:22,439 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:22,439 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:22,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:22,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:22,440 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:22,443 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/B5BED59F58C14F82A8A871DC0B8271C1/EE6AF1F45B564843847FFAA7F49709B6/.tez/application_1463534482149_0001
2016-05-18 01:21:22,444 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace/_temporary
2016-05-18 01:21:22,444 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.319
2016-05-18 01:21:22,496 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-05-18 01:21:22,537 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:22,538 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:22,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 01:21:22,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 01:21:22,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:22,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:22,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:22,634 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 01:21:22,644 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-18 01:21:22,705 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-18 01:21:22,705 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:22,705 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-18 01:21:22,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-18 01:21:22,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-18 01:21:22,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-18 01:21:22,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-05-18 01:21:22,707 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:22,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-18 01:21:22,724 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:22,725 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:22,725 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:22,733 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001 doesn't exist and is created
2016-05-18 01:21:22,753 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:22,753 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:22,754 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd
2016-05-18 01:21:22,770 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534482725_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:22,770 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:22,770 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:22,772 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:22,772 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:22,772 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:22,772 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:22,773 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:22,775 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:22,775 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:22,776 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:22,776 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:22,776 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:22,776 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:22,778 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:22,779 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:22,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:22,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-log-dir
2016-05-18 01:21:22,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-log-dir/history.txt.appattempt_1463534482725_0001_000000, maxErrors=10
2016-05-18 01:21:22,780 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:22,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534482725_0001_000000, appSubmitTime=1463534482770, launchTime=1463534482770
2016-05-18 01:21:22,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:22,789 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48838
2016-05-18 01:21:22,790 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534482725
2016-05-18 01:21:22,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534482725_0001_000000, startTime=1463534482791
2016-05-18 01:21:22,792 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:22,854 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:22,854 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:22,854 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463534482725_0001, dagName=[7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2
2016-05-18 01:21:22,872 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2
2016-05-18 01:21:22,872 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:22,873 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534482725_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-log-dir/dag_1463534482725_0001_1.dot
2016-05-18 01:21:22,873 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-log-dir/dag_1463534482725_0001_1-tez-dag.pb.txt
2016-05-18 01:21:22,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534482725_0001_1
2016-05-18 01:21:22,880 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,880 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2
2016-05-18 01:21:22 Running Dag: dag_1463534482725_0001_1
2016-05-18 01:21:22,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534482725_0001_1, submitTime=1463534482872
2016-05-18 01:21:22,895 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001/tez-conf.pb" } size: 40949 timestamp: 1463534482000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:22,895 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,895 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:22,896 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001/tez-conf.pb" } size: 40949 timestamp: 1463534482000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:22,896 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:22,896 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:22,896 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:22,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534482725_0001_1, initTime=1463534482897
2016-05-18 01:21:22,897 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482725_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:22,897 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:22,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:DAG_STARTED]: dagID=dag_1463534482725_0001_1, startTime=1463534482899
2016-05-18 01:21:22,899 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463534482725_0001, dagName=[7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2
2016-05-18 01:21:22,899 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482725_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A3B9F50D94D44746B2349F39822BD469 : {56F1D3D41FA549FB8F0D5156B658EBE3={InputName=56F1D3D41FA549FB8F0D5156B658EBE3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 56F1D3D41FA549FB8F0D5156B658EBE3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,900 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,901 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:22,901 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:22,904 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:22,904 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:22,904 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:22,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534482725_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:22,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B], numAdditionalOutputs=1
2016-05-18 01:21:22,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=FC3933D808CD46EB89DC0AFDA08DA44A, vertex=vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:22,907 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F28DD492A40C49FAA1D8F046F52FBB0B:FC3933D808CD46EB89DC0AFDA08DA44A using oldmapred API
2016-05-18 01:21:22,907 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=FC3933D808CD46EB89DC0AFDA08DA44A, vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:22,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, vertexId=vertex_1463534482725_0001_1_01, initRequestedTime=1463534482901, initedTime=1463534482905, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:22,909 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:22,908 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 56F1D3D41FA549FB8F0D5156B658EBE3 on vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,912 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 56F1D3D41FA549FB8F0D5156B658EBE3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:22,914 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:22,914 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 01:21:22,915 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:22,915 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:22,915 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 56F1D3D41FA549FB8F0D5156B658EBE3 on vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,916 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:22,916 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {56F1D3D41FA549FB8F0D5156B658EBE3=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:22,917 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] parallelism set to 1
2016-05-18 01:21:22,917 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:22,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3B9F50D94D44746B2349F39822BD469, vertexId=vertex_1463534482725_0001_1_00, initRequestedTime=1463534482900, initedTime=1463534482917, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:22,918 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] with distanceFromRoot: 0
2016-05-18 01:21:22,918 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:22,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534482725_0001_1_00, startRequestedTime=1463534482901, startedTime=1463534482918
2016-05-18 01:21:22,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534482725_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:22,919 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:22,919 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534482725_0001_1_00 for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] numStartedSources: 1 numSources: 1
2016-05-18 01:21:22,919 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] with distanceFromRoot: 1
2016-05-18 01:21:22,919 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A3B9F50D94D44746B2349F39822BD469
2016-05-18 01:21:22,920 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534482725_0001_1_01, startRequestedTime=1463534482919, startedTime=1463534482919
2016-05-18 01:21:22,921 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:22,924 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F28DD492A40C49FAA1D8F046F52FBB0B with 0 source tasks and 1 pending tasks
2016-05-18 01:21:22,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_STARTED]: vertexName=A3B9F50D94D44746B2349F39822BD469, taskId=task_1463534482725_0001_1_00_000000, scheduledTime=1463534482924, launchTime=1463534482924
2016-05-18 01:21:22,924 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A3B9F50D94D44746B2349F39822BD469 in vertex: F28DD492A40C49FAA1D8F046F52FBB0B numBipartiteSourceTasks: 1
2016-05-18 01:21:22,924 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:22,925 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:22,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534482725_0001_00_000001, launchTime=1463534482927
2016-05-18 01:21:22,927 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482725_0001_00_000001
2016-05-18 01:21:22,928 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482725_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,928 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534482725_0001_1_00_000000_0] started. Is using containerId: [container_1463534482725_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:22,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3B9F50D94D44746B2349F39822BD469, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0, startTime=1463534482928, containerId=container_1463534482725_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:22 Starting to run new task attempt: attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,929 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,929 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:22,929 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:22,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2, VertexName: A3B9F50D94D44746B2349F39822BD469, VertexParallelism: 1, TaskAttemptID:attempt_1463534482725_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=56F1D3D41FA549FB8F0D5156B658EBE3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F28DD492A40C49FAA1D8F046F52FBB0B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:22,929 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:22,930 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,933 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463534482725_0001, with dag id: dag_1463534482725_0001_1
2016-05-18 01:21:22,934 INFO input.MRInput (MRInput.java:initialize(440)) - 56F1D3D41FA549FB8F0D5156B658EBE3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:22,934 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 56F1D3D41FA549FB8F0D5156B658EBE3
2016-05-18 01:21:22,934 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:22,936 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534482725_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:22,936 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534482725_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:22,943 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3B9F50D94D44746B2349F39822BD469, ordinal: 0
2016-05-18 01:21:22,943 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3B9F50D94D44746B2349F39822BD469, mem on start (mb), free: 154, total: 376, max: 672
2016-05-18 01:21:22,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:22,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:22,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:22,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:22,944 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[56F1D3D41FA549FB8F0D5156B658EBE3:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F28DD492A40C49FAA1D8F046F52FBB0B:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:22,944 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:22,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:22,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:22,946 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[56F1D3D41FA549FB8F0D5156B658EBE3:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F28DD492A40C49FAA1D8F046F52FBB0B:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:22,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:22,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 56F1D3D41FA549FB8F0D5156B658EBE3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:22,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 56F1D3D41FA549FB8F0D5156B658EBE3
2016-05-18 01:21:22,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:22,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:22,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:22,949 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,950 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 01:21:22,950 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 56F1D3D41FA549FB8F0D5156B658EBE3 initialized RecordReader from event
2016-05-18 01:21:22,953 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:22,954 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 56F1D3D41FA549FB8F0D5156B658EBE3
2016-05-18 01:21:22,954 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: BEDC91DEDAC34D7688CDFBD366C513B1
2016-05-18 01:21:22,955 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] BEDC91DEDAC34D7688CDFBD366C513B1
2016-05-18 01:21:22,957 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F28DD492A40C49FAA1D8F046F52FBB0B using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@35887164, comparator=cascading.tuple.hadoop.util.TupleComparator@10cc77b1, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:22,957 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F28DD492A40C49FAA1D8F046F52FBB0B: , UsingHashComparator=false
2016-05-18 01:21:22,986 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:22,986 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:22,986 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F28DD492A40C49FAA1D8F046F52FBB0B: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:22,986 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F28DD492A40C49FAA1D8F046F52FBB0B: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:22,987 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:22,987 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3B9F50D94D44746B2349F39822BD469, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:22,991 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,991 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3B9F50D94D44746B2349F39822BD469, mem on close (mb), free: 237, total: 375, max: 672
2016-05-18 01:21:22,991 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F28DD492A40C49FAA1D8F046F52FBB0B: Starting flush of map output
2016-05-18 01:21:22,991 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F28DD492A40C49FAA1D8F046F52FBB0B: Span0.length = 20, perItem = 23
2016-05-18 01:21:22,993 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F28DD492A40C49FAA1D8F046F52FBB0B: done sorting span=0, length=20, time=1
2016-05-18 01:21:22,993 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F28DD492A40C49FAA1D8F046F52FBB0B: Heap = SpanIterator<0:19> (span=Span[16777216,466]),
2016-05-18 01:21:22,994 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F28DD492A40C49FAA1D8F046F52FBB0B: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-local-dir/output/attempt_1463534482725_0001_1_00_000000_0_10068_0/file.out
2016-05-18 01:21:22,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:22 Completed running task attempt: attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:22,998 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:22,998 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534482725_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:22,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534482725_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534482988, Process_Duration=3, Process_End_Time=1463534482991, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 01:21:23,000 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534482725_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3B9F50D94D44746B2349F39822BD469, taskAttemptId=attempt_1463534482725_0001_1_00_000000_0, creationTime=1463534482924, allocationTime=1463534482926, startTime=1463534482928, finishTime=1463534483000, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:23,000 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,000 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:23,001 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,001 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534482725_0001_1_00_000000_0
2016-05-18 01:21:23,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_FINISHED]: vertexName=A3B9F50D94D44746B2349F39822BD469, taskId=task_1463534482725_0001_1_00_000000, startTime=1463534482928, finishTime=1463534483001, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1463534482725_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534482988, Process_Duration=3, Process_End_Time=1463534482991, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 01:21:23,002 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482725_0001_00_000001
2016-05-18 01:21:23,002 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:23,003 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:23,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_FINISHED]: vertexName=A3B9F50D94D44746B2349F39822BD469, vertexId=vertex_1463534482725_0001_1_00, initRequestedTime=1463534482900, initedTime=1463534482917, startRequestedTime=1463534482901, startedTime=1463534482918, finishTime=1463534483003, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534482988, Process_Duration=3, Process_End_Time=1463534482991, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463534482928, firstTasksToStart=[ task_1463534482725_0001_1_00_000000 ], lastTaskFinishTime=1463534483000, lastTasksToFinish=[ task_1463534482725_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534482725_0001_1_00_000000 ], longestDurationTasks=[ task_1463534482725_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:23,004 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:23,004 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] attempt: attempt_1463534482725_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:23,004 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F28DD492A40C49FAA1D8F046F52FBB0B
2016-05-18 01:21:23,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534482725_0001_00_000001, stoppedTime=1463534483004, exitStatus=0
2016-05-18 01:21:23,004 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A3B9F50D94D44746B2349F39822BD469, destinationVertexName=F28DD492A40C49FAA1D8F046F52FBB0B. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:23,005 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:23,004 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:23,005 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,006 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534482725_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:23,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_STARTED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, taskId=task_1463534482725_0001_1_01_000000, scheduledTime=1463534483006, launchTime=1463534483006
2016-05-18 01:21:23,007 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:23,007 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534482725_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:23,008 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482725_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:23,008 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534482725_0001_00_000001, Exiting
2016-05-18 01:21:23,008 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534482725_0001_00_000001
2016-05-18 01:21:23,008 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534482725_0001_00_000001
2016-05-18 01:21:23,010 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534482725_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:23,010 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534482725_0001_00_000001 completed successfully
2016-05-18 01:21:23,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534482725_0001_00_000002, launchTime=1463534483011
2016-05-18 01:21:23,012 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482725_0001_00_000002
2016-05-18 01:21:23,012 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482725_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,013 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534482725_0001_1_01_000000_0] started. Is using containerId: [container_1463534482725_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:23 Starting to run new task attempt: attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,013 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,013 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0, startTime=1463534483012, containerId=container_1463534482725_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:23,014 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:23,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2, VertexName: F28DD492A40C49FAA1D8F046F52FBB0B, VertexParallelism: 1, TaskAttemptID:attempt_1463534482725_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A3B9F50D94D44746B2349F39822BD469, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FC3933D808CD46EB89DC0AFDA08DA44A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:23,014 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:23,015 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534482725_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:23,016 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534482725_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:23,015 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,019 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:23,020 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:23,031 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F28DD492A40C49FAA1D8F046F52FBB0B, ordinal: 1
2016-05-18 01:21:23,032 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F28DD492A40C49FAA1D8F046F52FBB0B, mem on start (mb), free: 231, total: 375, max: 672
2016-05-18 01:21:23,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:23,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:23,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:23,035 INFO output.MROutput (MROutput.java:initialize(421)) - FC3933D808CD46EB89DC0AFDA08DA44A: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:23,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:23,035 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A3B9F50D94D44746B2349F39822BD469:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [FC3933D808CD46EB89DC0AFDA08DA44A:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:23,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:23,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:23,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:23,036 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A3B9F50D94D44746B2349F39822BD469:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [FC3933D808CD46EB89DC0AFDA08DA44A:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:23,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:23,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A3B9F50D94D44746B2349F39822BD469 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:23,037 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A3B9F50D94D44746B2349F39822BD469: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:23,037 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A3B9F50D94D44746B2349F39822BD469: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:23,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:23,038 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A3B9F50D94D44746B2349F39822BD469 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:23,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:23,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A3B9F50D94D44746B2349F39822BD469
2016-05-18 01:21:23,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:23,042 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,047 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-18 01:21:23,047 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A3B9F50D94D44746B2349F39822BD469: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:23,047 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: BEDC91DEDAC34D7688CDFBD366C513B1
2016-05-18 01:21:23,048 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: FC3933D808CD46EB89DC0AFDA08DA44A
2016-05-18 01:21:23,048 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-05-18 01:21:23,048 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] BEDC91DEDAC34D7688CDFBD366C513B1, for 1 inputs
2016-05-18 01:21:23,050 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534482725_0001_1_00_000000_0_10068} to DISK_DIRECT, csize=463, dsize=459, EndTime=1463534483050, TimeTaken=1, Rate=0.44 MB/s
2016-05-18 01:21:23,050 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A3B9F50D94D44746B2349F39822BD469
2016-05-18 01:21:23,050 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:23,050 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A3B9F50D94D44746B2349F39822BD469, wasInterrupted=false
2016-05-18 01:21:23,051 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:23,051 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:23,052 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001_wd/localmode-local-dir/output/attempt_1463534482725_0001_1_00_000000_0_10068/file.out, len=463, isLocal=true
2016-05-18 01:21:23,052 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 463 bytes from disk
2016-05-18 01:21:23,052 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:23,052 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:23,052 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-18 01:21:23,052 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A3B9F50D94D44746B2349F39822BD469
2016-05-18 01:21:23,053 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A3B9F50D94D44746B2349F39822BD469: Shuffle Runner thread complete
2016-05-18 01:21:23,053 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F28DD492A40C49FAA1D8F046F52FBB0B, all 1 inputs ready in: 00:00:00.004
2016-05-18 01:21:23,053 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A3B9F50D94D44746B2349F39822BD469: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:23,054 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - A3B9F50D94D44746B2349F39822BD469: Already shutdown. Ignoring fetch complete
2016-05-18 01:21:23,060 INFO output.MROutput (MROutput.java:close(526)) - FC3933D808CD46EB89DC0AFDA08DA44A closed
2016-05-18 01:21:23,060 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534482725_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:23,061 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,061 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F28DD492A40C49FAA1D8F046F52FBB0B, mem on close (mb), free: 228, total: 375, max: 672
2016-05-18 01:21:23,061 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A3B9F50D94D44746B2349F39822BD469
2016-05-18 01:21:23,061 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A3B9F50D94D44746B2349F39822BD469: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:23,061 INFO output.MROutput (MROutput.java:close(526)) - FC3933D808CD46EB89DC0AFDA08DA44A closed
2016-05-18 01:21:23,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:23 Completed running task attempt: attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,062 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:23,062 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534482725_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:23,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534482725_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534483053, Process_Duration=8, Process_End_Time=1463534483061, Read_Duration=4, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-18 01:21:23,064 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534482725_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,064 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,064 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:23,064 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,064 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534482725_0001_00_000002
2016-05-18 01:21:23,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, taskAttemptId=attempt_1463534482725_0001_1_01_000000_0, creationTime=1463534483007, allocationTime=1463534483008, startTime=1463534483012, finishTime=1463534483064, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:23,065 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534482725_0001_1_01_000000_0
2016-05-18 01:21:23,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:TASK_FINISHED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, taskId=task_1463534482725_0001_1_01_000000, startTime=1463534483012, finishTime=1463534483065, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463534482725_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534483053, Process_Duration=8, Process_End_Time=1463534483061, Read_Duration=4, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-18 01:21:23,066 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:23,066 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B]
2016-05-18 01:21:23,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:VERTEX_FINISHED]: vertexName=F28DD492A40C49FAA1D8F046F52FBB0B, vertexId=vertex_1463534482725_0001_1_01, initRequestedTime=1463534482901, initedTime=1463534482905, startRequestedTime=1463534482919, startedTime=1463534482919, finishTime=1463534483066, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534483053, Process_Duration=8, Process_End_Time=1463534483061, Read_Duration=4, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463534483012, firstTasksToStart=[ task_1463534482725_0001_1_01_000000 ], lastTaskFinishTime=1463534483064, lastTasksToFinish=[ task_1463534482725_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534482725_0001_1_01_000000 ], longestDurationTasks=[ task_1463534482725_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:23,067 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:23,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534482725_0001_00_000002, stoppedTime=1463534483067, exitStatus=0
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534482725_0001_1_00 [A3B9F50D94D44746B2349F39822BD469]
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534482725_0001_1
2016-05-18 01:21:23,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534482725_0001_1
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482725_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:23,069 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534482725_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:23,068 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: FC3933D808CD46EB89DC0AFDA08DA44A for vertex: vertex_1463534482725_0001_1_01 [F28DD492A40C49FAA1D8F046F52FBB0B], outputName: FC3933D808CD46EB89DC0AFDA08DA44A
2016-05-18 01:21:23,075 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534482725_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:23,076 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534482725_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:23,076 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534482725_0001_00_000002, Exiting
2016-05-18 01:21:23,076 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534482725_0001_00_000002
2016-05-18 01:21:23,077 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534482725_0001_00_000002
2016-05-18 01:21:23,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534482725_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:23,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534482725_0001_00_000002 completed successfully
2016-05-18 01:21:23,080 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:FC3933D808CD46EB89DC0AFDA08DA44A of vertex/vertexGroup:F28DD492A40C49FAA1D8F046F52FBB0B isVertexGroupOutput:false
2016-05-18 01:21:23,080 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,082 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534482725_0001_1, queueSize=0
2016-05-18 01:21:23,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534482725_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534482725_0001_1, startTime=1463534482897, finishTime=1463534483080, timeTaken=183, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=370, AM_GC_TIME_MILLIS=15, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068966041, Process_Duration=11, Process_End_Time=2927068966052, Read_Duration=5, Tuples_Read=40, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-18 01:21:23,083 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534482725_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:23,083 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534482725_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:23 Completed Dag: dag_1463534482725_0001_1
2016-05-18 01:21:23,083 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534482725_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:23,084 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534482725_0001_1_post
2016-05-18 01:21:23,084 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,084 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:23,084 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:23,084 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2, with id=dag_1463534482725_0001_1
2016-05-18 01:21:23,084 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A] count/(1/1) ...latform/testCount2/count2, with id=dag_1463534482725_0001_1
2016-05-18 01:21:23,092 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1463534482725_0001
2016-05-18 01:21:23,095 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:23,095 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:23,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:23,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:23,096 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:23,098 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:23,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:23,098 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:23,099 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:23,099 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:23,099 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:23,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:23,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:23,100 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:23,103 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7F11E842538342BF99F649C6F0739D40/C725584FAEF449C596C018B35505620A/.tez/application_1463534482725_0001
2016-05-18 01:21:23,104 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2/_temporary
2016-05-18 01:21:23,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.398
2016-05-18 01:21:23,198 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:23,199 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:23,208 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-05-18 01:21:23,240 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:23,240 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:23,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 01:21:23,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 01:21:23,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:23,299 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:23,299 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:23,323 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 01:21:23,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-18 01:21:23,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-18 01:21:23,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:23,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-18 01:21:23,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-18 01:21:23,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-18 01:21:23,392 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-18 01:21:23,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-05-18 01:21:23,393 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:23,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-18 01:21:23,411 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:23,412 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:23,412 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:23,422 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001 doesn't exist and is created
2016-05-18 01:21:23,442 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:23,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:23,444 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd
2016-05-18 01:21:23,462 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534483412_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:23,463 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:23,463 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:23,464 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:23,464 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:23,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:23,465 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:23,466 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:23,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:23,467 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:23,468 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:23,468 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:23,469 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:23,470 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:23,471 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:23,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:23,472 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-log-dir
2016-05-18 01:21:23,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-log-dir/history.txt.appattempt_1463534483412_0001_000000, maxErrors=10
2016-05-18 01:21:23,473 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:23,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534483412_0001_000000, appSubmitTime=1463534483462, launchTime=1463534483462
2016-05-18 01:21:23,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:23,487 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40123
2016-05-18 01:21:23,487 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534483412
2016-05-18 01:21:23,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534483412_0001_000000, startTime=1463534483489
2016-05-18 01:21:23,490 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:23,543 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:23,543 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:23,544 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463534483412_0001, dagName=[66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3
2016-05-18 01:21:23,563 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3
2016-05-18 01:21:23,564 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:23,565 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534483412_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-log-dir/dag_1463534483412_0001_1.dot
2016-05-18 01:21:23,565 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-log-dir/dag_1463534483412_0001_1-tez-dag.pb.txt
2016-05-18 01:21:23,572 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534483412_0001_1
2016-05-18 01:21:23,572 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,572 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3
2016-05-18 01:21:23 Running Dag: dag_1463534483412_0001_1
2016-05-18 01:21:23,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534483412_0001_1, submitTime=1463534483563
2016-05-18 01:21:23,588 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001/tez-conf.pb" } size: 40949 timestamp: 1463534483000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:23,588 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,589 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:23,590 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001/tez-conf.pb" } size: 40949 timestamp: 1463534483000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:23,590 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,590 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:23,591 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:23,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534483412_0001_1, initTime=1463534483591
2016-05-18 01:21:23,591 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534483412_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:23,592 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:23,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:DAG_STARTED]: dagID=dag_1463534483412_0001_1, startTime=1463534483592
2016-05-18 01:21:23,593 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534483412_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:23,593 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AA6AD345EDB64565A5C91B0D856B5221 : {897E8283C27645C28694881DB6B41C37={InputName=897E8283C27645C28694881DB6B41C37}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:23,593 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 897E8283C27645C28694881DB6B41C37, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:23,593 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,593 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,593 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,594 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,594 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463534483412_0001, dagName=[66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3
2016-05-18 01:21:23,594 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:23,594 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,594 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 897E8283C27645C28694881DB6B41C37 on vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,597 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 897E8283C27645C28694881DB6B41C37 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:23,597 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:23,598 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,598 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534483412_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:23,599 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A], numAdditionalOutputs=1
2016-05-18 01:21:23,599 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8B9A6E5C07244655BC8FE5D6EE1862BA, vertex=vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:23,600 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:23,600 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 01:21:23,601 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:23,601 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:23,602 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 897E8283C27645C28694881DB6B41C37 on vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,602 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D34D42D08F464E13B61044900DC6717A:8B9A6E5C07244655BC8FE5D6EE1862BA using oldmapred API
2016-05-18 01:21:23,602 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8B9A6E5C07244655BC8FE5D6EE1862BA, vertexName=D34D42D08F464E13B61044900DC6717A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:23,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D34D42D08F464E13B61044900DC6717A, vertexId=vertex_1463534483412_0001_1_01, initRequestedTime=1463534483594, initedTime=1463534483599, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:23,604 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:23,605 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:23,606 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {897E8283C27645C28694881DB6B41C37=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:23,606 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] parallelism set to 1
2016-05-18 01:21:23,606 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:23,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, vertexId=vertex_1463534483412_0001_1_00, initRequestedTime=1463534483593, initedTime=1463534483607, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:23,607 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] with distanceFromRoot: 0
2016-05-18 01:21:23,607 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:23,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534483412_0001_1_00, startRequestedTime=1463534483594, startedTime=1463534483607
2016-05-18 01:21:23,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534483412_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:23,608 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:23,608 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534483412_0001_1_00 for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] numStartedSources: 1 numSources: 1
2016-05-18 01:21:23,608 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] with distanceFromRoot: 1
2016-05-18 01:21:23,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534483412_0001_1_01, startRequestedTime=1463534483608, startedTime=1463534483608
2016-05-18 01:21:23,608 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:23,611 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D34D42D08F464E13B61044900DC6717A with 0 source tasks and 1 pending tasks
2016-05-18 01:21:23,612 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AA6AD345EDB64565A5C91B0D856B5221
2016-05-18 01:21:23,613 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,613 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: AA6AD345EDB64565A5C91B0D856B5221 in vertex: D34D42D08F464E13B61044900DC6717A numBipartiteSourceTasks: 1
2016-05-18 01:21:23,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_STARTED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, taskId=task_1463534483412_0001_1_00_000000, scheduledTime=1463534483613, launchTime=1463534483613
2016-05-18 01:21:23,614 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,614 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:23,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534483412_0001_00_000001, launchTime=1463534483616
2016-05-18 01:21:23,616 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534483412_0001_00_000001
2016-05-18 01:21:23,617 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534483412_0001_1_00_000000_0] started. Is using containerId: [container_1463534483412_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:23,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0, startTime=1463534483617, containerId=container_1463534483412_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:23,618 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534483412_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23 Starting to run new task attempt: attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,619 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,620 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:23,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3, VertexName: AA6AD345EDB64565A5C91B0D856B5221, VertexParallelism: 1, TaskAttemptID:attempt_1463534483412_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=897E8283C27645C28694881DB6B41C37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D34D42D08F464E13B61044900DC6717A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:23,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463534483412_0001, with dag id: dag_1463534483412_0001_1
2016-05-18 01:21:23,621 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:23,623 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,625 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534483412_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:23,627 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534483412_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:23,628 INFO input.MRInput (MRInput.java:initialize(440)) - 897E8283C27645C28694881DB6B41C37 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:23,628 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 897E8283C27645C28694881DB6B41C37
2016-05-18 01:21:23,645 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:23,650 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AA6AD345EDB64565A5C91B0D856B5221, ordinal: 0
2016-05-18 01:21:23,650 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA6AD345EDB64565A5C91B0D856B5221, mem on start (mb), free: 334, total: 372, max: 672
2016-05-18 01:21:23,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:23,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:23,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:23,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:23,651 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[897E8283C27645C28694881DB6B41C37:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D34D42D08F464E13B61044900DC6717A:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:23,651 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:23,651 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:23,652 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:23,652 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[897E8283C27645C28694881DB6B41C37:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D34D42D08F464E13B61044900DC6717A:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:23,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:23,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 897E8283C27645C28694881DB6B41C37 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:23,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 897E8283C27645C28694881DB6B41C37
2016-05-18 01:21:23,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:23,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:23,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:23,654 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,655 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 01:21:23,656 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 897E8283C27645C28694881DB6B41C37 initialized RecordReader from event
2016-05-18 01:21:23,658 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:23,659 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 897E8283C27645C28694881DB6B41C37
2016-05-18 01:21:23,659 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: C0834437D8844C39AFC5E11875C39C0C
2016-05-18 01:21:23,660 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] C0834437D8844C39AFC5E11875C39C0C
2016-05-18 01:21:23,661 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D34D42D08F464E13B61044900DC6717A using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@2f813e50, comparator=cascading.tuple.hadoop.util.TupleComparator@4e220e8d, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:23,662 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D34D42D08F464E13B61044900DC6717A: , UsingHashComparator=false
2016-05-18 01:21:23,676 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:23,676 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:23,677 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D34D42D08F464E13B61044900DC6717A: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:23,677 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D34D42D08F464E13B61044900DC6717A: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:23,677 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:23,678 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AA6AD345EDB64565A5C91B0D856B5221, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:23,680 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,681 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA6AD345EDB64565A5C91B0D856B5221, mem on close (mb), free: 232, total: 372, max: 672
2016-05-18 01:21:23,681 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D34D42D08F464E13B61044900DC6717A: Starting flush of map output
2016-05-18 01:21:23,681 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D34D42D08F464E13B61044900DC6717A: Span0.length = 20, perItem = 23
2016-05-18 01:21:23,684 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D34D42D08F464E13B61044900DC6717A: done sorting span=0, length=20, time=3
2016-05-18 01:21:23,684 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D34D42D08F464E13B61044900DC6717A: Heap = SpanIterator<0:19> (span=Span[16777216,466]),
2016-05-18 01:21:23,686 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D34D42D08F464E13B61044900DC6717A: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-local-dir/output/attempt_1463534483412_0001_1_00_000000_0_10074_0/file.out
2016-05-18 01:21:23,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:23 Completed running task attempt: attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,689 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:23,690 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534483412_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:23,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534483412_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534483678, Process_Duration=2, Process_End_Time=1463534483680, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 01:21:23,691 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534483412_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,691 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, taskAttemptId=attempt_1463534483412_0001_1_00_000000_0, creationTime=1463534483614, allocationTime=1463534483615, startTime=1463534483617, finishTime=1463534483691, timeTaken=74, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:23,691 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:23,692 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,692 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534483412_0001_1_00_000000_0
2016-05-18 01:21:23,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_FINISHED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, taskId=task_1463534483412_0001_1_00_000000, startTime=1463534483617, finishTime=1463534483692, timeTaken=75, status=SUCCEEDED, successfulAttemptID=attempt_1463534483412_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534483678, Process_Duration=2, Process_End_Time=1463534483680, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 01:21:23,693 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534483412_0001_00_000001
2016-05-18 01:21:23,693 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:23,693 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_FINISHED]: vertexName=AA6AD345EDB64565A5C91B0D856B5221, vertexId=vertex_1463534483412_0001_1_00, initRequestedTime=1463534483593, initedTime=1463534483607, startRequestedTime=1463534483594, startedTime=1463534483607, finishTime=1463534483693, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534483678, Process_Duration=2, Process_End_Time=1463534483680, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463534483617, firstTasksToStart=[ task_1463534483412_0001_1_00_000000 ], lastTaskFinishTime=1463534483691, lastTasksToFinish=[ task_1463534483412_0001_1_00_000000 ], minTaskDuration=74, maxTaskDuration=74, avgTaskDuration=74.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534483412_0001_1_00_000000 ], longestDurationTasks=[ task_1463534483412_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:23,695 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:23,695 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] attempt: attempt_1463534483412_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:23,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534483412_0001_00_000001, stoppedTime=1463534483695, exitStatus=0
2016-05-18 01:21:23,695 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D34D42D08F464E13B61044900DC6717A
2016-05-18 01:21:23,695 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=AA6AD345EDB64565A5C91B0D856B5221, destinationVertexName=D34D42D08F464E13B61044900DC6717A. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:23,695 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,695 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:23,696 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,696 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534483412_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:23,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_STARTED]: vertexName=D34D42D08F464E13B61044900DC6717A, taskId=task_1463534483412_0001_1_01_000000, scheduledTime=1463534483696, launchTime=1463534483696
2016-05-18 01:21:23,696 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534483412_0001_00_000002, launchTime=1463534483698
2016-05-18 01:21:23,698 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534483412_0001_00_000002
2016-05-18 01:21:23,699 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534483412_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,699 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534483412_0001_1_01_000000_0] started. Is using containerId: [container_1463534483412_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:23,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D34D42D08F464E13B61044900DC6717A, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0, startTime=1463534483699, containerId=container_1463534483412_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:23 Starting to run new task attempt: attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,700 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,700 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,700 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:23,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3, VertexName: D34D42D08F464E13B61044900DC6717A, VertexParallelism: 1, TaskAttemptID:attempt_1463534483412_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AA6AD345EDB64565A5C91B0D856B5221, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8B9A6E5C07244655BC8FE5D6EE1862BA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:23,700 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:23,701 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,701 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534483412_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,702 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534483412_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:23,703 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534483412_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:23,703 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534483412_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:23,703 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534483412_0001_00_000001, Exiting
2016-05-18 01:21:23,704 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534483412_0001_00_000001
2016-05-18 01:21:23,704 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534483412_0001_00_000001
2016-05-18 01:21:23,705 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534483412_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:23,706 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534483412_0001_00_000001 completed successfully
2016-05-18 01:21:23,706 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:23,711 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:23,716 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D34D42D08F464E13B61044900DC6717A, ordinal: 1
2016-05-18 01:21:23,716 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D34D42D08F464E13B61044900DC6717A, mem on start (mb), free: 226, total: 372, max: 672
2016-05-18 01:21:23,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:23,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:23,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:23,716 INFO output.MROutput (MROutput.java:initialize(421)) - 8B9A6E5C07244655BC8FE5D6EE1862BA: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:23,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:23,717 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8B9A6E5C07244655BC8FE5D6EE1862BA:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [AA6AD345EDB64565A5C91B0D856B5221:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:23,717 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:23,717 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:23,718 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:23,718 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8B9A6E5C07244655BC8FE5D6EE1862BA:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [AA6AD345EDB64565A5C91B0D856B5221:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:23,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:23,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: AA6AD345EDB64565A5C91B0D856B5221 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:23,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:23,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:23,719 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - AA6AD345EDB64565A5C91B0D856B5221: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:23,720 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - AA6AD345EDB64565A5C91B0D856B5221: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:23,722 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: AA6AD345EDB64565A5C91B0D856B5221 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:23,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: AA6AD345EDB64565A5C91B0D856B5221
2016-05-18 01:21:23,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:23,725 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,726 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AA6AD345EDB64565A5C91B0D856B5221: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:23,727 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534483412_0001_1_00_000000_0_10074} to DISK_DIRECT, csize=463, dsize=459, EndTime=1463534483727, TimeTaken=1, Rate=0.44 MB/s
2016-05-18 01:21:23,728 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : AA6AD345EDB64565A5C91B0D856B5221
2016-05-18 01:21:23,728 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:23,728 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: AA6AD345EDB64565A5C91B0D856B5221, wasInterrupted=false
2016-05-18 01:21:23,729 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:23,730 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-18 01:21:23,730 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:23,730 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001_wd/localmode-local-dir/output/attempt_1463534483412_0001_1_00_000000_0_10074/file.out, len=463, isLocal=true
2016-05-18 01:21:23,730 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: C0834437D8844C39AFC5E11875C39C0C
2016-05-18 01:21:23,730 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: 8B9A6E5C07244655BC8FE5D6EE1862BA
2016-05-18 01:21:23,731 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-05-18 01:21:23,730 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 463 bytes from disk
2016-05-18 01:21:23,731 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] C0834437D8844C39AFC5E11875C39C0C, for 1 inputs
2016-05-18 01:21:23,731 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:23,732 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:23,734 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-18 01:21:23,734 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : AA6AD345EDB64565A5C91B0D856B5221
2016-05-18 01:21:23,734 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D34D42D08F464E13B61044900DC6717A, all 1 inputs ready in: 00:00:00.003
2016-05-18 01:21:23,735 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - AA6AD345EDB64565A5C91B0D856B5221: Shuffle Runner thread complete
2016-05-18 01:21:23,735 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - AA6AD345EDB64565A5C91B0D856B5221: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:23,739 INFO output.MROutput (MROutput.java:close(526)) - 8B9A6E5C07244655BC8FE5D6EE1862BA closed
2016-05-18 01:21:23,740 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534483412_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:23,740 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,740 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D34D42D08F464E13B61044900DC6717A, mem on close (mb), free: 224, total: 372, max: 672
2016-05-18 01:21:23,740 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: AA6AD345EDB64565A5C91B0D856B5221
2016-05-18 01:21:23,740 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AA6AD345EDB64565A5C91B0D856B5221: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:23,741 INFO output.MROutput (MROutput.java:close(526)) - 8B9A6E5C07244655BC8FE5D6EE1862BA closed
2016-05-18 01:21:23,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:23 Completed running task attempt: attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,741 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:23,741 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534483412_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:23,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534483412_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534483735, Process_Duration=5, Process_End_Time=1463534483740, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-18 01:21:23,743 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534483412_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D34D42D08F464E13B61044900DC6717A, taskAttemptId=attempt_1463534483412_0001_1_01_000000_0, creationTime=1463534483696, allocationTime=1463534483697, startTime=1463534483699, finishTime=1463534483743, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:23,743 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534483412_0001_1_01_000000_0
2016-05-18 01:21:23,743 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:23,744 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:23,744 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:TASK_FINISHED]: vertexName=D34D42D08F464E13B61044900DC6717A, taskId=task_1463534483412_0001_1_01_000000, startTime=1463534483699, finishTime=1463534483743, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1463534483412_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534483735, Process_Duration=5, Process_End_Time=1463534483740, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17
2016-05-18 01:21:23,744 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534483412_0001_00_000002
2016-05-18 01:21:23,744 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:23,744 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A]
2016-05-18 01:21:23,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:VERTEX_FINISHED]: vertexName=D34D42D08F464E13B61044900DC6717A, vertexId=vertex_1463534483412_0001_1_01, initRequestedTime=1463534483594, initedTime=1463534483599, startRequestedTime=1463534483608, startedTime=1463534483608, finishTime=1463534483744, timeTaken=136, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534483735, Process_Duration=5, Process_End_Time=1463534483740, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463534483699, firstTasksToStart=[ task_1463534483412_0001_1_01_000000 ], lastTaskFinishTime=1463534483743, lastTasksToFinish=[ task_1463534483412_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534483412_0001_1_01_000000 ], longestDurationTasks=[ task_1463534483412_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:23,745 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:23,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534483412_0001_00_000002, stoppedTime=1463534483745, exitStatus=0
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534483412_0001_1_00 [AA6AD345EDB64565A5C91B0D856B5221]
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534483412_0001_1
2016-05-18 01:21:23,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534483412_0001_1
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534483412_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:23,746 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8B9A6E5C07244655BC8FE5D6EE1862BA for vertex: vertex_1463534483412_0001_1_01 [D34D42D08F464E13B61044900DC6717A], outputName: 8B9A6E5C07244655BC8FE5D6EE1862BA
2016-05-18 01:21:23,747 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534483412_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:23,755 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534483412_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:23,755 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534483412_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:23,755 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534483412_0001_00_000002, Exiting
2016-05-18 01:21:23,755 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534483412_0001_00_000002
2016-05-18 01:21:23,755 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534483412_0001_00_000002
2016-05-18 01:21:23,755 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8B9A6E5C07244655BC8FE5D6EE1862BA of vertex/vertexGroup:D34D42D08F464E13B61044900DC6717A isVertexGroupOutput:false
2016-05-18 01:21:23,755 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:23,755 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534483412_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:23,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534483412_0001_00_000002 completed successfully
2016-05-18 01:21:23,758 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534483412_0001_1, queueSize=0
2016-05-18 01:21:23,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534483412_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534483412_0001_1, startTime=1463534483592, finishTime=1463534483755, timeTaken=163, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=290, AM_GC_TIME_MILLIS=15, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068967413, Process_Duration=7, Process_End_Time=2927068967420, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-18 01:21:23,758 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534483412_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:23,758 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534483412_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:23 Completed Dag: dag_1463534483412_0001_1
2016-05-18 01:21:23,759 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534483412_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:23,759 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534483412_0001_1_post
2016-05-18 01:21:23,759 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:23,759 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:23,759 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:23,759 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3, with id=dag_1463534483412_0001_1
2016-05-18 01:21:23,760 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB] count/(1/1) ...latform/testCount3/count3, with id=dag_1463534483412_0001_1
2016-05-18 01:21:23,767 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1463534483412_0001
2016-05-18 01:21:23,769 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:23,769 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:23,769 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:23,769 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:23,770 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:23,770 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:23,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:23,772 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:23,772 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:23,772 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:23,773 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:23,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:23,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:23,775 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:23,778 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/66D3079187474592BECA14BC8E940D25/A6DF7B5772BB4895879D254F43F739DB/.tez/application_1463534483412_0001
2016-05-18 01:21:23,779 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3/_temporary
2016-05-18 01:21:23,779 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.387
2016-05-18 01:21:23,855 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-05-18 01:21:23,880 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:23,881 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:23,919 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:21:23,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 01:21:23,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:23,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:23,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:23,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 01:21:23,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-18 01:21:24,006 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting
2016-05-18 01:21:24,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:24,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-18 01:21:24,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] parallel execution of steps is enabled: false
2016-05-18 01:21:24,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] executing total steps: 1
2016-05-18 01:21:24,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] allocating management threads: 1
2016-05-18 01:21:24,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-05-18 01:21:24,009 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:24,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] tez session mode enabled: true
2016-05-18 01:21:24,028 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:24,028 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:24,029 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:24,038 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001 doesn't exist and is created
2016-05-18 01:21:24,058 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:24,058 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:24,059 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd
2016-05-18 01:21:24,075 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534484028_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:24,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:24,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:24,076 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:24,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:24,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:24,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:24,078 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:24,079 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:24,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:24,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:24,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:24,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:24,080 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:24,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:24,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:24,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:24,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:24,082 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:24,082 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:24,083 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:24,084 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:24,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:24,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-log-dir
2016-05-18 01:21:24,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-log-dir/history.txt.appattempt_1463534484028_0001_000000, maxErrors=10
2016-05-18 01:21:24,085 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:24,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534484028_0001_000000, appSubmitTime=1463534484074, launchTime=1463534484074
2016-05-18 01:21:24,088 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:24,092 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35300
2016-05-18 01:21:24,093 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534484028
2016-05-18 01:21:24,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534484028_0001_000000, startTime=1463534484094
2016-05-18 01:21:24,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:24,158 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:24,159 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:24,159 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463534484028_0001, dagName=[AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4
2016-05-18 01:21:24,176 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4
2016-05-18 01:21:24,176 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:24,177 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534484028_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-log-dir/dag_1463534484028_0001_1.dot
2016-05-18 01:21:24,178 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-log-dir/dag_1463534484028_0001_1-tez-dag.pb.txt
2016-05-18 01:21:24,184 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534484028_0001_1
2016-05-18 01:21:24,185 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,185 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4
2016-05-18 01:21:24 Running Dag: dag_1463534484028_0001_1
2016-05-18 01:21:24,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534484028_0001_1, submitTime=1463534484176
2016-05-18 01:21:24,201 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001/tez-conf.pb" } size: 40949 timestamp: 1463534484000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:24,201 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,201 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:24,202 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001/tez-conf.pb" } size: 40949 timestamp: 1463534484000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:24,202 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,203 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:24,203 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:24,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534484028_0001_1, initTime=1463534484203
2016-05-18 01:21:24,204 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484028_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:24,204 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:24,205 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:DAG_STARTED]: dagID=dag_1463534484028_0001_1, startTime=1463534484204
2016-05-18 01:21:24,205 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484028_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:24,205 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F31289914E15474CB7845438167CBD47 : {154865A4EC0F40C287766D5191A2AB8B={InputName=154865A4EC0F40C287766D5191A2AB8B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:24,205 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 154865A4EC0F40C287766D5191A2AB8B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:24,205 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,206 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,206 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,206 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,207 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:24,207 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 154865A4EC0F40C287766D5191A2AB8B on vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,207 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,207 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463534484028_0001, dagName=[AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4
2016-05-18 01:21:24,209 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 154865A4EC0F40C287766D5191A2AB8B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:24,211 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:24,211 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,211 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534484028_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:24,212 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2], numAdditionalOutputs=1
2016-05-18 01:21:24,212 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8A7228973769443599099E69616E0415, vertex=vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:24,212 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:24,212 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 01:21:24,212 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:24,213 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:24,213 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 154865A4EC0F40C287766D5191A2AB8B on vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,214 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B49B4FD98C72414C8DF1908B2E9E39F2:8A7228973769443599099E69616E0415 using oldmapred API
2016-05-18 01:21:24,214 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8A7228973769443599099E69616E0415, vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:24,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, vertexId=vertex_1463534484028_0001_1_01, initRequestedTime=1463534484207, initedTime=1463534484212, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:24,215 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:24,217 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:24,217 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {154865A4EC0F40C287766D5191A2AB8B=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:24,218 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] parallelism set to 1
2016-05-18 01:21:24,218 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:24,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F31289914E15474CB7845438167CBD47, vertexId=vertex_1463534484028_0001_1_00, initRequestedTime=1463534484205, initedTime=1463534484218, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:24,218 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] with distanceFromRoot: 0
2016-05-18 01:21:24,219 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:24,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534484028_0001_1_00, startRequestedTime=1463534484207, startedTime=1463534484219
2016-05-18 01:21:24,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534484028_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:24,219 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:24,219 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534484028_0001_1_00 for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] numStartedSources: 1 numSources: 1
2016-05-18 01:21:24,219 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] with distanceFromRoot: 1
2016-05-18 01:21:24,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534484028_0001_1_01, startRequestedTime=1463534484219, startedTime=1463534484220
2016-05-18 01:21:24,221 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:24,223 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: B49B4FD98C72414C8DF1908B2E9E39F2 with 0 source tasks and 1 pending tasks
2016-05-18 01:21:24,224 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: F31289914E15474CB7845438167CBD47 in vertex: B49B4FD98C72414C8DF1908B2E9E39F2 numBipartiteSourceTasks: 1
2016-05-18 01:21:24,224 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F31289914E15474CB7845438167CBD47
2016-05-18 01:21:24,225 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_STARTED]: vertexName=F31289914E15474CB7845438167CBD47, taskId=task_1463534484028_0001_1_00_000000, scheduledTime=1463534484225, launchTime=1463534484225
2016-05-18 01:21:24,239 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,240 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:24,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534484028_0001_00_000001, launchTime=1463534484242
2016-05-18 01:21:24,243 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484028_0001_00_000001
2016-05-18 01:21:24,243 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:24,244 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:24,244 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534484028_0001_1_00_000000_0] started. Is using containerId: [container_1463534484028_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:24,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F31289914E15474CB7845438167CBD47, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0, startTime=1463534484243, containerId=container_1463534484028_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:24,243 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484028_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24 Starting to run new task attempt: attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,246 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,247 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,247 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:24,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4, VertexName: F31289914E15474CB7845438167CBD47, VertexParallelism: 1, TaskAttemptID:attempt_1463534484028_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=154865A4EC0F40C287766D5191A2AB8B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B49B4FD98C72414C8DF1908B2E9E39F2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:24,248 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:24,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] submitted tez dag to app master: application_1463534484028_0001, with dag id: dag_1463534484028_0001_1
2016-05-18 01:21:24,249 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,257 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534484028_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:24,257 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534484028_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:24,259 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:24,262 INFO input.MRInput (MRInput.java:initialize(440)) - 154865A4EC0F40C287766D5191A2AB8B using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:24,262 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 154865A4EC0F40C287766D5191A2AB8B
2016-05-18 01:21:24,270 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F31289914E15474CB7845438167CBD47, ordinal: 0
2016-05-18 01:21:24,270 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F31289914E15474CB7845438167CBD47, mem on start (mb), free: 325, total: 371, max: 672
2016-05-18 01:21:24,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:24,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:24,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:24,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:24,272 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[154865A4EC0F40C287766D5191A2AB8B:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [B49B4FD98C72414C8DF1908B2E9E39F2:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:24,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:24,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:24,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:24,274 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[154865A4EC0F40C287766D5191A2AB8B:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [B49B4FD98C72414C8DF1908B2E9E39F2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:24,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:24,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 154865A4EC0F40C287766D5191A2AB8B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:24,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:24,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:24,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 154865A4EC0F40C287766D5191A2AB8B
2016-05-18 01:21:24,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:24,278 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,284 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:24,284 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 01:21:24,285 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 154865A4EC0F40C287766D5191A2AB8B initialized RecordReader from event
2016-05-18 01:21:24,285 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 154865A4EC0F40C287766D5191A2AB8B
2016-05-18 01:21:24,285 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0790C628525A49019B50E17D4A6C7ADB
2016-05-18 01:21:24,285 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0790C628525A49019B50E17D4A6C7ADB
2016-05-18 01:21:24,287 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - B49B4FD98C72414C8DF1908B2E9E39F2 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4d99acd, comparator=cascading.tuple.hadoop.util.TupleComparator@6a420b37, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:24,287 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for B49B4FD98C72414C8DF1908B2E9E39F2: , UsingHashComparator=false
2016-05-18 01:21:24,300 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:24,300 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:24,300 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for B49B4FD98C72414C8DF1908B2E9E39F2: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:24,300 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - B49B4FD98C72414C8DF1908B2E9E39F2: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:24,301 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 01:21:24,301 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F31289914E15474CB7845438167CBD47, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:24,304 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,304 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F31289914E15474CB7845438167CBD47, mem on close (mb), free: 223, total: 371, max: 672
2016-05-18 01:21:24,304 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - B49B4FD98C72414C8DF1908B2E9E39F2: Starting flush of map output
2016-05-18 01:21:24,305 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - B49B4FD98C72414C8DF1908B2E9E39F2: Span0.length = 20, perItem = 23
2016-05-18 01:21:24,306 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - B49B4FD98C72414C8DF1908B2E9E39F2: done sorting span=0, length=20, time=1
2016-05-18 01:21:24,306 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - B49B4FD98C72414C8DF1908B2E9E39F2: Heap = SpanIterator<0:19> (span=Span[16777216,466]),
2016-05-18 01:21:24,308 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - B49B4FD98C72414C8DF1908B2E9E39F2: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-local-dir/output/attempt_1463534484028_0001_1_00_000000_0_10080_0/file.out
2016-05-18 01:21:24,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:24 Completed running task attempt: attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,311 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:24,312 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534484028_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:24,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534484028_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534484301, Process_Duration=3, Process_End_Time=1463534484304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 01:21:24,313 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534484028_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F31289914E15474CB7845438167CBD47, taskAttemptId=attempt_1463534484028_0001_1_00_000000_0, creationTime=1463534484239, allocationTime=1463534484241, startTime=1463534484243, finishTime=1463534484313, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:24,314 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534484028_0001_1_00_000000_0
2016-05-18 01:21:24,313 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_FINISHED]: vertexName=F31289914E15474CB7845438167CBD47, taskId=task_1463534484028_0001_1_00_000000, startTime=1463534484243, finishTime=1463534484314, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463534484028_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534484301, Process_Duration=3, Process_End_Time=1463534484304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 01:21:24,314 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:24,314 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:24,315 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,315 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484028_0001_00_000001
2016-05-18 01:21:24,315 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_FINISHED]: vertexName=F31289914E15474CB7845438167CBD47, vertexId=vertex_1463534484028_0001_1_00, initRequestedTime=1463534484205, initedTime=1463534484218, startRequestedTime=1463534484207, startedTime=1463534484219, finishTime=1463534484315, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534484301, Process_Duration=3, Process_End_Time=1463534484304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463534484243, firstTasksToStart=[ task_1463534484028_0001_1_00_000000 ], lastTaskFinishTime=1463534484313, lastTasksToFinish=[ task_1463534484028_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534484028_0001_1_00_000000 ], longestDurationTasks=[ task_1463534484028_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:24,316 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:24,317 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] attempt: attempt_1463534484028_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:24,317 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B49B4FD98C72414C8DF1908B2E9E39F2
2016-05-18 01:21:24,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534484028_0001_00_000001, stoppedTime=1463534484317, exitStatus=0
2016-05-18 01:21:24,317 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=F31289914E15474CB7845438167CBD47, destinationVertexName=B49B4FD98C72414C8DF1908B2E9E39F2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:24,317 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,317 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:24,318 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:24,319 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534484028_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:24,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_STARTED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, taskId=task_1463534484028_0001_1_01_000000, scheduledTime=1463534484319, launchTime=1463534484319
2016-05-18 01:21:24,320 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534484028_0001_00_000002, launchTime=1463534484321
2016-05-18 01:21:24,322 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484028_0001_00_000002
2016-05-18 01:21:24,322 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534484028_0001_1_01_000000_0] started. Is using containerId: [container_1463534484028_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:24,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0, startTime=1463534484322, containerId=container_1463534484028_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:24,323 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484028_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24 Starting to run new task attempt: attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,323 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,323 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,324 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:24,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4, VertexName: B49B4FD98C72414C8DF1908B2E9E39F2, VertexParallelism: 1, TaskAttemptID:attempt_1463534484028_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F31289914E15474CB7845438167CBD47, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8A7228973769443599099E69616E0415, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:24,324 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:24,324 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,325 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534484028_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,325 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534484028_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:24,326 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534484028_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:24,326 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484028_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:24,326 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534484028_0001_00_000001, Exiting
2016-05-18 01:21:24,326 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534484028_0001_00_000001
2016-05-18 01:21:24,326 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534484028_0001_00_000001
2016-05-18 01:21:24,328 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:24,329 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:24,331 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534484028_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:24,331 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534484028_0001_00_000001 completed successfully
2016-05-18 01:21:24,337 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B49B4FD98C72414C8DF1908B2E9E39F2, ordinal: 1
2016-05-18 01:21:24,337 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B49B4FD98C72414C8DF1908B2E9E39F2, mem on start (mb), free: 218, total: 371, max: 672
2016-05-18 01:21:24,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:24,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:24,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:24,338 INFO output.MROutput (MROutput.java:initialize(421)) - 8A7228973769443599099E69616E0415: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:24,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:24,339 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8A7228973769443599099E69616E0415:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [F31289914E15474CB7845438167CBD47:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 01:21:24,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:24,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:24,339 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:24,340 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8A7228973769443599099E69616E0415:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [F31289914E15474CB7845438167CBD47:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 01:21:24,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:24,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F31289914E15474CB7845438167CBD47 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:24,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:24,340 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - F31289914E15474CB7845438167CBD47: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:24,341 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - F31289914E15474CB7845438167CBD47: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:24,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:24,342 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: F31289914E15474CB7845438167CBD47 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:24,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F31289914E15474CB7845438167CBD47
2016-05-18 01:21:24,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:24,345 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,345 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F31289914E15474CB7845438167CBD47: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:24,348 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534484028_0001_1_00_000000_0_10080} to DISK_DIRECT, csize=463, dsize=459, EndTime=1463534484348, TimeTaken=1, Rate=0.44 MB/s
2016-05-18 01:21:24,348 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : F31289914E15474CB7845438167CBD47
2016-05-18 01:21:24,348 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:24,348 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: F31289914E15474CB7845438167CBD47, wasInterrupted=false
2016-05-18 01:21:24,349 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-05-18 01:21:24,350 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:24,350 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0790C628525A49019B50E17D4A6C7ADB
2016-05-18 01:21:24,350 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: 8A7228973769443599099E69616E0415
2016-05-18 01:21:24,351 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-05-18 01:21:24,351 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0790C628525A49019B50E17D4A6C7ADB, for 1 inputs
2016-05-18 01:21:24,352 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:24,353 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001_wd/localmode-local-dir/output/attempt_1463534484028_0001_1_00_000000_0_10080/file.out, len=463, isLocal=true
2016-05-18 01:21:24,353 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 463 bytes from disk
2016-05-18 01:21:24,353 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:24,353 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:24,353 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-05-18 01:21:24,354 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : F31289914E15474CB7845438167CBD47
2016-05-18 01:21:24,354 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - F31289914E15474CB7845438167CBD47: Shuffle Runner thread complete
2016-05-18 01:21:24,354 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B49B4FD98C72414C8DF1908B2E9E39F2, all 1 inputs ready in: 00:00:00.003
2016-05-18 01:21:24,354 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - F31289914E15474CB7845438167CBD47: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:24,359 INFO output.MROutput (MROutput.java:close(526)) - 8A7228973769443599099E69616E0415 closed
2016-05-18 01:21:24,359 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534484028_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:24,360 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,360 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B49B4FD98C72414C8DF1908B2E9E39F2, mem on close (mb), free: 216, total: 371, max: 672
2016-05-18 01:21:24,361 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: F31289914E15474CB7845438167CBD47
2016-05-18 01:21:24,361 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F31289914E15474CB7845438167CBD47: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:24,361 INFO output.MROutput (MROutput.java:close(526)) - 8A7228973769443599099E69616E0415 closed
2016-05-18 01:21:24,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:24 Completed running task attempt: attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,361 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:24,362 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534484028_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:24,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534484028_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534484354, Process_Duration=6, Process_End_Time=1463534484360, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-05-18 01:21:24,363 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534484028_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,363 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, taskAttemptId=attempt_1463534484028_0001_1_01_000000_0, creationTime=1463534484320, allocationTime=1463534484321, startTime=1463534484322, finishTime=1463534484363, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:24,364 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534484028_0001_1_01_000000_0
2016-05-18 01:21:24,364 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:24,364 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,365 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484028_0001_00_000002
2016-05-18 01:21:24,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:TASK_FINISHED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, taskId=task_1463534484028_0001_1_01_000000, startTime=1463534484322, finishTime=1463534484364, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463534484028_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534484354, Process_Duration=6, Process_End_Time=1463534484360, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-05-18 01:21:24,365 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:24,365 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2]
2016-05-18 01:21:24,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:VERTEX_FINISHED]: vertexName=B49B4FD98C72414C8DF1908B2E9E39F2, vertexId=vertex_1463534484028_0001_1_01, initRequestedTime=1463534484207, initedTime=1463534484212, startRequestedTime=1463534484219, startedTime=1463534484220, finishTime=1463534484365, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534484354, Process_Duration=6, Process_End_Time=1463534484360, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1463534484322, firstTasksToStart=[ task_1463534484028_0001_1_01_000000 ], lastTaskFinishTime=1463534484363, lastTasksToFinish=[ task_1463534484028_0001_1_01_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534484028_0001_1_01_000000 ], longestDurationTasks=[ task_1463534484028_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:24,366 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:24,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534484028_0001_00_000002, stoppedTime=1463534484367, exitStatus=0
2016-05-18 01:21:24,367 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:24,367 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:24,367 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534484028_0001_1_00 [F31289914E15474CB7845438167CBD47]
2016-05-18 01:21:24,367 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534484028_0001_1
2016-05-18 01:21:24,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534484028_0001_1
2016-05-18 01:21:24,368 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484028_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:24,368 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8A7228973769443599099E69616E0415 for vertex: vertex_1463534484028_0001_1_01 [B49B4FD98C72414C8DF1908B2E9E39F2], outputName: 8A7228973769443599099E69616E0415
2016-05-18 01:21:24,368 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534484028_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:24,368 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534484028_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:24,368 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484028_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:24,368 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534484028_0001_00_000002, Exiting
2016-05-18 01:21:24,369 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534484028_0001_00_000002
2016-05-18 01:21:24,369 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534484028_0001_00_000002
2016-05-18 01:21:24,369 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534484028_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:24,369 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534484028_0001_00_000002 completed successfully
2016-05-18 01:21:24,380 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8A7228973769443599099E69616E0415 of vertex/vertexGroup:B49B4FD98C72414C8DF1908B2E9E39F2 isVertexGroupOutput:false
2016-05-18 01:21:24,380 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:24,383 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534484028_0001_1, queueSize=0
2016-05-18 01:21:24,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484028_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534484028_0001_1, startTime=1463534484204, finishTime=1463534484380, timeTaken=176, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=410, AM_GC_TIME_MILLIS=14, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=17, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068968655, Process_Duration=9, Process_End_Time=2927068968664, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-05-18 01:21:24,384 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534484028_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:24,384 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484028_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:24 Completed Dag: dag_1463534484028_0001_1
2016-05-18 01:21:24,384 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534484028_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:24,384 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534484028_0001_1_post
2016-05-18 01:21:24,384 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,384 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:24,384 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:24,385 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4, with id=dag_1463534484028_0001_1
2016-05-18 01:21:24,385 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121] count/(1/1) ...latform/testCount4/count4, with id=dag_1463534484028_0001_1
2016-05-18 01:21:24,390 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1463534484028_0001
2016-05-18 01:21:24,392 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:24,392 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:24,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:24,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:24,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:24,394 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:24,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:24,395 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:24,395 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:24,395 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:24,396 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:24,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:24,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:24,398 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:24,401 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/AADBF973E3724CB38F2847AD285CE329/E49FEB6A683041B391D45FE36B34B121/.tez/application_1463534484028_0001
2016-05-18 01:21:24,401 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4/_temporary
2016-05-18 01:21:24,402 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [count] completed in: 00:00.395
2016-05-18 01:21:24,490 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-05-18 01:21:24,516 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:24,516 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:24,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:21:24,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-18 01:21:24,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:24,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 01:21:24,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:24,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:24,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-18 01:21:24,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:24,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:24,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-18 01:21:24,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:24,589 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:24,589 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:24,591 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-05-18 01:21:24,591 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:24,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:24,609 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:24,610 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:24,610 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:24,619 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001 doesn't exist and is created
2016-05-18 01:21:24,640 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:24,640 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:24,642 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001_wd
2016-05-18 01:21:24,657 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534484610_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:24,657 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:24,657 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:24,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:24,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:24,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:24,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:24,660 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:24,661 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:24,661 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:24,662 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:24,662 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:24,662 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:24,662 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:24,662 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:24,663 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:24,663 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:24,663 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:24,663 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:24,663 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:24,665 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:24,666 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:24,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:24,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001_wd/localmode-log-dir
2016-05-18 01:21:24,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001_wd/localmode-log-dir/history.txt.appattempt_1463534484610_0001_000000, maxErrors=10
2016-05-18 01:21:24,667 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:24,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534484610_0001_000000, appSubmitTime=1463534484656, launchTime=1463534484656
2016-05-18 01:21:24,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:24,672 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46978
2016-05-18 01:21:24,675 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534484610
2016-05-18 01:21:24,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534484610_0001_000000, startTime=1463534484678
2016-05-18 01:21:24,678 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:24,741 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:24,741 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:24,741 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463534484610_0001, dagName=[7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter
2016-05-18 01:21:24,757 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter
2016-05-18 01:21:24,757 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:24,758 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534484610_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001_wd/localmode-log-dir/dag_1463534484610_0001_1.dot
2016-05-18 01:21:24,759 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001_wd/localmode-log-dir/dag_1463534484610_0001_1-tez-dag.pb.txt
2016-05-18 01:21:24,762 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534484610_0001_1
2016-05-18 01:21:24,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,763 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter
2016-05-18 01:21:24 Running Dag: dag_1463534484610_0001_1
2016-05-18 01:21:24,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534484610_0001_1, submitTime=1463534484757
2016-05-18 01:21:24,777 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001/tez-conf.pb" } size: 40949 timestamp: 1463534484000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:24,777 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,778 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,778 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:24,778 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:24,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534484610_0001_1, initTime=1463534484779
2016-05-18 01:21:24,779 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484610_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:24,780 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:24,780 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463534484610_0001, dagName=[7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter
2016-05-18 01:21:24,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:DAG_STARTED]: dagID=dag_1463534484610_0001_1, startTime=1463534484780
2016-05-18 01:21:24,780 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484610_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:24,780 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AFB5A2A0E7B44F03B93A31E4746CE519 : {ADAB331DB7C24C45B21FB013E236EDAF={InputName=ADAB331DB7C24C45B21FB013E236EDAF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:24,780 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: ADAB331DB7C24C45B21FB013E236EDAF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:24,780 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,781 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,781 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,781 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,782 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:24,782 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: ADAB331DB7C24C45B21FB013E236EDAF on vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,784 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input ADAB331DB7C24C45B21FB013E236EDAF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:24,785 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:24,786 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:24,786 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:24,786 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:24,787 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: ADAB331DB7C24C45B21FB013E236EDAF on vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,788 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:24,788 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {ADAB331DB7C24C45B21FB013E236EDAF=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:24,789 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] parallelism set to 1
2016-05-18 01:21:24,789 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:24,789 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519], numAdditionalOutputs=1
2016-05-18 01:21:24,789 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=73A5B5B52AC9456E8155AAC28EB83979, vertex=vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:24,791 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AFB5A2A0E7B44F03B93A31E4746CE519:73A5B5B52AC9456E8155AAC28EB83979 using oldmapred API
2016-05-18 01:21:24,792 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=73A5B5B52AC9456E8155AAC28EB83979, vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:24,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, vertexId=vertex_1463534484610_0001_1_00, initRequestedTime=1463534484780, initedTime=1463534484789, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:24,793 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] with distanceFromRoot: 0
2016-05-18 01:21:24,793 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:24,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534484610_0001_1_00, startRequestedTime=1463534484782, startedTime=1463534484793
2016-05-18 01:21:24,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534484610_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:24,794 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:24,794 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AFB5A2A0E7B44F03B93A31E4746CE519
2016-05-18 01:21:24,795 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:TASK_STARTED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, taskId=task_1463534484610_0001_1_00_000000, scheduledTime=1463534484795, launchTime=1463534484795
2016-05-18 01:21:24,796 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,797 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:24,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534484610_0001_00_000001, launchTime=1463534484798
2016-05-18 01:21:24,799 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484610_0001_00_000001
2016-05-18 01:21:24,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534484610_0001_1_00_000000_0] started. Is using containerId: [container_1463534484610_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:24,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0, startTime=1463534484799, containerId=container_1463534484610_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:24,800 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484610_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24 Starting to run new task attempt: attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,801 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,801 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,802 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:24,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter, VertexName: AFB5A2A0E7B44F03B93A31E4746CE519, VertexParallelism: 1, TaskAttemptID:attempt_1463534484610_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADAB331DB7C24C45B21FB013E236EDAF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=73A5B5B52AC9456E8155AAC28EB83979, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:24,802 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:24,803 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,803 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534484610_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:24,803 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534484610_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:24,806 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534484610_0001, with dag id: dag_1463534484610_0001_1
2016-05-18 01:21:24,807 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:24,811 INFO input.MRInput (MRInput.java:initialize(440)) - ADAB331DB7C24C45B21FB013E236EDAF using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:24,811 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: ADAB331DB7C24C45B21FB013E236EDAF
2016-05-18 01:21:24,816 INFO output.MROutput (MROutput.java:initialize(421)) - 73A5B5B52AC9456E8155AAC28EB83979: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:24,817 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AFB5A2A0E7B44F03B93A31E4746CE519, ordinal: 0
2016-05-18 01:21:24,817 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AFB5A2A0E7B44F03B93A31E4746CE519, mem on start (mb), free: 129, total: 371, max: 672
2016-05-18 01:21:24,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:24,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:24,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:24,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:24,819 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[73A5B5B52AC9456E8155AAC28EB83979:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [ADAB331DB7C24C45B21FB013E236EDAF:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 01:21:24,819 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:24,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:24,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 01:21:24,821 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[73A5B5B52AC9456E8155AAC28EB83979:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [ADAB331DB7C24C45B21FB013E236EDAF:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 01:21:24,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:24,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: ADAB331DB7C24C45B21FB013E236EDAF
2016-05-18 01:21:24,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: ADAB331DB7C24C45B21FB013E236EDAF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:24,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:24,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:24,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:24,824 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,825 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:24,826 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - ADAB331DB7C24C45B21FB013E236EDAF initialized RecordReader from event
2016-05-18 01:21:24,828 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:24,828 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: ADAB331DB7C24C45B21FB013E236EDAF
2016-05-18 01:21:24,829 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: 73A5B5B52AC9456E8155AAC28EB83979
2016-05-18 01:21:24,829 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-05-18 01:21:24,830 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:24,830 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AFB5A2A0E7B44F03B93A31E4746CE519, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:24,832 INFO output.MROutput (MROutput.java:close(526)) - 73A5B5B52AC9456E8155AAC28EB83979 closed
2016-05-18 01:21:24,833 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534484610_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 01:21:24,833 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,834 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AFB5A2A0E7B44F03B93A31E4746CE519, mem on close (mb), free: 127, total: 371, max: 672
2016-05-18 01:21:24,834 INFO output.MROutput (MROutput.java:close(526)) - 73A5B5B52AC9456E8155AAC28EB83979 closed
2016-05-18 01:21:24,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:24 Completed running task attempt: attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,835 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:24,835 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534484610_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:24,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534484610_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1463534484830, Process_Duration=3, Process_End_Time=1463534484833, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-05-18 01:21:24,836 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534484610_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,836 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:24,836 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:24,836 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, taskAttemptId=attempt_1463534484610_0001_1_00_000000_0, creationTime=1463534484796, allocationTime=1463534484797, startTime=1463534484799, finishTime=1463534484836, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:24,837 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534484610_0001_1_00_000000_0
2016-05-18 01:21:24,837 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534484610_0001_00_000001
2016-05-18 01:21:24,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:TASK_FINISHED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, taskId=task_1463534484610_0001_1_00_000000, startTime=1463534484799, finishTime=1463534484837, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463534484610_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463534484830, Process_Duration=3, Process_End_Time=1463534484833, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-18 01:21:24,838 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:24,838 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519]
2016-05-18 01:21:24,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:VERTEX_FINISHED]: vertexName=AFB5A2A0E7B44F03B93A31E4746CE519, vertexId=vertex_1463534484610_0001_1_00, initRequestedTime=1463534484780, initedTime=1463534484789, startRequestedTime=1463534484782, startedTime=1463534484793, finishTime=1463534484838, timeTaken=45, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463534484830, Process_Duration=3, Process_End_Time=1463534484833, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1463534484799, firstTasksToStart=[ task_1463534484610_0001_1_00_000000 ], lastTaskFinishTime=1463534484836, lastTasksToFinish=[ task_1463534484610_0001_1_00_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534484610_0001_1_00_000000 ], longestDurationTasks=[ task_1463534484610_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:24,839 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:24,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534484610_0001_00_000001, stoppedTime=1463534484839, exitStatus=0
2016-05-18 01:21:24,839 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 01:21:24,839 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:24,839 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534484610_0001_1
2016-05-18 01:21:24,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534484610_0001_1
2016-05-18 01:21:24,840 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 73A5B5B52AC9456E8155AAC28EB83979 for vertex: vertex_1463534484610_0001_1_00 [AFB5A2A0E7B44F03B93A31E4746CE519], outputName: 73A5B5B52AC9456E8155AAC28EB83979
2016-05-18 01:21:24,841 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484610_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:24,841 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534484610_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:24,848 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:73A5B5B52AC9456E8155AAC28EB83979 of vertex/vertexGroup:AFB5A2A0E7B44F03B93A31E4746CE519 isVertexGroupOutput:false
2016-05-18 01:21:24,848 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 01:21:24,849 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534484610_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:24,849 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534484610_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:24,849 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534484610_0001_00_000001, Exiting
2016-05-18 01:21:24,849 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534484610_0001_00_000001
2016-05-18 01:21:24,849 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534484610_0001_00_000001
2016-05-18 01:21:24,850 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534484610_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:24,850 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534484610_0001_1, queueSize=0
2016-05-18 01:21:24,850 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534484610_0001_00_000001 completed successfully
2016-05-18 01:21:24,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534484610_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534484610_0001_1, startTime=1463534484780, finishTime=1463534484848, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=110, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1463534484830, Process_Duration=3, Process_End_Time=1463534484833, Read_Duration=1, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-05-18 01:21:24,851 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534484610_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:24,851 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534484610_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:24 Completed Dag: dag_1463534484610_0001_1
2016-05-18 01:21:24,851 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534484610_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:24,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534484610_0001_1_post
2016-05-18 01:21:24,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:24,852 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:24,852 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:24,853 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter, with id=dag_1463534484610_0001_1
2016-05-18 01:21:24,853 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9] test/(1/1) ...latform/testFilter/filter, with id=dag_1463534484610_0001_1
2016-05-18 01:21:24,858 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1463534484610_0001
2016-05-18 01:21:24,859 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:24,859 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:24,859 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:24,859 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:24,860 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:24,860 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:24,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:24,861 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:24,861 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:24,861 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:24,861 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:24,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:24,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:24,862 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:24,865 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/7186F07453194ADABD929F2B205455F6/AEBEEE0B16514848B6F3B6AF42BB78E9/.tez/application_1463534484610_0001
2016-05-18 01:21:24,866 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter/_temporary
2016-05-18 01:21:24,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.277
2016-05-18 01:21:24,917 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-05-18 01:21:24,949 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 01:21:24,949 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 01:21:24,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 01:21:25,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 01:21:25,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:25,002 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 01:21:25,003 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 01:21:25,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt
2016-05-18 01:21:25,026 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-05-18 01:21:25,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-18 01:21:25,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:25,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-18 01:21:25,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-18 01:21:25,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-18 01:21:25,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-18 01:21:25,082 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 01:21:25,082 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 01:21:25,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-05-18 01:21:25,082 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 01:21:25,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-18 01:21:25,101 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:25,102 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 01:21:25,102 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 01:21:25,111 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001 doesn't exist and is created
2016-05-18 01:21:25,130 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 01:21:25,130 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 01:21:25,131 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd
2016-05-18 01:21:25,147 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463534485102_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 01:21:25,147 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:25,147 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2016-05-18 01:21:25,149 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 01:21:25,149 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 01:21:25,149 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 01:21:25,149 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 01:21:25,152 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 01:21:25,154 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 01:21:25,154 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 01:21:25,155 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 01:21:25,155 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 01:21:25,156 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 01:21:25,156 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 01:21:25,156 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 01:21:25,156 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 01:21:25,156 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:25,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 01:21:25,157 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 01:21:25,157 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 01:21:25,158 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 01:21:25,159 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 01:21:25,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 01:21:25,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-log-dir
2016-05-18 01:21:25,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-log-dir/history.txt.appattempt_1463534485102_0001_000000, maxErrors=10
2016-05-18 01:21:25,161 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 01:21:25,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463534485102_0001_000000, appSubmitTime=1463534485146, launchTime=1463534485147
2016-05-18 01:21:25,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 01:21:25,170 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33434
2016-05-18 01:21:25,170 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463534485102
2016-05-18 01:21:25,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463534485102_0001_000000, startTime=1463534485172
2016-05-18 01:21:25,172 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 01:21:25,231 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 01:21:25,231 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 01:21:25,231 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463534485102_0001, dagName=[BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple
2016-05-18 01:21:25,252 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple
2016-05-18 01:21:25,253 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 01:21:25,254 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463534485102_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-log-dir/dag_1463534485102_0001_1.dot
2016-05-18 01:21:25,255 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-log-dir/dag_1463534485102_0001_1-tez-dag.pb.txt
2016-05-18 01:21:25,271 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534485102_0001_1
2016-05-18 01:21:25,271 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,271 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple
2016-05-18 01:21:25 Running Dag: dag_1463534485102_0001_1
2016-05-18 01:21:25,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463534485102_0001_1, submitTime=1463534485252
2016-05-18 01:21:25,290 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001/tez-conf.pb" } size: 40949 timestamp: 1463534485000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:25,290 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,290 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:25,291 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001/tez-conf.pb" } size: 40949 timestamp: 1463534485000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E], Default Resources=<memory:1024, vCores:1>
2016-05-18 01:21:25,291 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,291 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 01:21:25,291 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 01:21:25,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463534485102_0001_1, initTime=1463534485292
2016-05-18 01:21:25,293 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534485102_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 01:21:25,294 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 01:21:25,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:DAG_STARTED]: dagID=dag_1463534485102_0001_1, startTime=1463534485294
2016-05-18 01:21:25,294 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534485102_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 01:21:25,294 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: CAB17C293498431DADC6938D5B1F365B : {54116925AC4A4C75A1BF55DE1EBD8113={InputName=54116925AC4A4C75A1BF55DE1EBD8113}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 01:21:25,294 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 54116925AC4A4C75A1BF55DE1EBD8113, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 01:21:25,294 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,294 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,294 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,295 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,295 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 01:21:25,295 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 54116925AC4A4C75A1BF55DE1EBD8113 on vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,295 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,297 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 54116925AC4A4C75A1BF55DE1EBD8113 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 01:21:25,299 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 01:21:25,299 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 1449 Min splitLength: 52428800 New desired splits: 1 Total length: 1449 Original splits: 2
2016-05-18 01:21:25,299 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 1449 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 1449 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 01:21:25,299 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 01:21:25,299 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 01:21:25,299 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,299 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 54116925AC4A4C75A1BF55DE1EBD8113 on vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,299 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534485102_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 01:21:25,300 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E], numAdditionalOutputs=1
2016-05-18 01:21:25,300 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=4A468014CF374C83B41DD966B2CE37DB, vertex=vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 01:21:25,300 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463534485102_0001, dagName=[BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple
2016-05-18 01:21:25,303 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DA7372BB0EB745608A147A2C1C7D051E:4A468014CF374C83B41DD966B2CE37DB using oldmapred API
2016-05-18 01:21:25,303 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=4A468014CF374C83B41DD966B2CE37DB, vertexName=DA7372BB0EB745608A147A2C1C7D051E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 01:21:25,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, vertexId=vertex_1463534485102_0001_1_01, initRequestedTime=1463534485295, initedTime=1463534485300, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 01:21:25,304 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] transitioned from NEW to INITED due to event V_INIT
2016-05-18 01:21:25,305 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:25,305 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {54116925AC4A4C75A1BF55DE1EBD8113=forAllWorkUnits=true, update=[1]}
2016-05-18 01:21:25,306 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] parallelism set to 1
2016-05-18 01:21:25,306 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 01:21:25,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAB17C293498431DADC6938D5B1F365B, vertexId=vertex_1463534485102_0001_1_00, initRequestedTime=1463534485294, initedTime=1463534485306, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 01:21:25,307 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] with distanceFromRoot: 0
2016-05-18 01:21:25,307 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 01:21:25,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534485102_0001_1_00, startRequestedTime=1463534485295, startedTime=1463534485307
2016-05-18 01:21:25,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463534485102_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 01:21:25,307 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:25,307 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463534485102_0001_1_00 for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] numStartedSources: 1 numSources: 1
2016-05-18 01:21:25,307 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] with distanceFromRoot: 1
2016-05-18 01:21:25,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463534485102_0001_1_01, startRequestedTime=1463534485307, startedTime=1463534485308
2016-05-18 01:21:25,308 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] transitioned from INITED to RUNNING due to event V_START
2016-05-18 01:21:25,311 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: DA7372BB0EB745608A147A2C1C7D051E with 0 source tasks and 1 pending tasks
2016-05-18 01:21:25,312 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in CAB17C293498431DADC6938D5B1F365B
2016-05-18 01:21:25,312 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,312 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: CAB17C293498431DADC6938D5B1F365B in vertex: DA7372BB0EB745608A147A2C1C7D051E numBipartiteSourceTasks: 1
2016-05-18 01:21:25,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_STARTED]: vertexName=CAB17C293498431DADC6938D5B1F365B, taskId=task_1463534485102_0001_1_00_000000, scheduledTime=1463534485313, launchTime=1463534485313
2016-05-18 01:21:25,313 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,314 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 01:21:25,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534485102_0001_00_000001, launchTime=1463534485315
2016-05-18 01:21:25,316 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534485102_0001_00_000001
2016-05-18 01:21:25,317 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534485102_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,317 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534485102_0001_1_00_000000_0] started. Is using containerId: [container_1463534485102_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 01:21:25,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAB17C293498431DADC6938D5B1F365B, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0, startTime=1463534485317, containerId=container_1463534485102_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 01:21:25 Starting to run new task attempt: attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,318 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,318 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,318 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:25,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple, VertexName: CAB17C293498431DADC6938D5B1F365B, VertexParallelism: 1, TaskAttemptID:attempt_1463534485102_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=54116925AC4A4C75A1BF55DE1EBD8113, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DA7372BB0EB745608A147A2C1C7D051E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 01:21:25,318 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:25,319 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463534485102_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 01:21:25,319 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534485102_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 01:21:25,319 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,323 INFO input.MRInput (MRInput.java:initialize(440)) - 54116925AC4A4C75A1BF55DE1EBD8113 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 01:21:25,323 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 54116925AC4A4C75A1BF55DE1EBD8113
2016-05-18 01:21:25,326 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:25,330 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463534485102_0001, with dag id: dag_1463534485102_0001_1
2016-05-18 01:21:25,335 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAB17C293498431DADC6938D5B1F365B, ordinal: 0
2016-05-18 01:21:25,335 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB17C293498431DADC6938D5B1F365B, mem on start (mb), free: 239, total: 370, max: 672
2016-05-18 01:21:25,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:25,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:25,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:25,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:25,336 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[54116925AC4A4C75A1BF55DE1EBD8113:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DA7372BB0EB745608A147A2C1C7D051E:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 01:21:25,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:25,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:25,337 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 01:21:25,337 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[54116925AC4A4C75A1BF55DE1EBD8113:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DA7372BB0EB745608A147A2C1C7D051E:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 01:21:25,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:25,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 54116925AC4A4C75A1BF55DE1EBD8113 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:25,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 54116925AC4A4C75A1BF55DE1EBD8113
2016-05-18 01:21:25,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:25,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:25,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:25,338 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,340 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:0+724, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt:724+725], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=1449}
2016-05-18 01:21:25,340 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 54116925AC4A4C75A1BF55DE1EBD8113 initialized RecordReader from event
2016-05-18 01:21:25,341 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:25,342 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 54116925AC4A4C75A1BF55DE1EBD8113
2016-05-18 01:21:25,342 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 95D73A26B5CE4EB69A2FB4723B471E22
2016-05-18 01:21:25,342 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 95D73A26B5CE4EB69A2FB4723B471E22
2016-05-18 01:21:25,344 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - DA7372BB0EB745608A147A2C1C7D051E using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@193bd0c6, comparator=cascading.tuple.hadoop.util.TupleComparator@2607841b, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 01:21:25,344 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for DA7372BB0EB745608A147A2C1C7D051E: , UsingHashComparator=false
2016-05-18 01:21:25,358 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 01:21:25,358 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 01:21:25,358 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for DA7372BB0EB745608A147A2C1C7D051E: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 01:21:25,358 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - DA7372BB0EB745608A147A2C1C7D051E: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 01:21:25,358 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/classes/test/data/apache.10.txt"]
2016-05-18 01:21:25,359 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAB17C293498431DADC6938D5B1F365B, all 1 inputs ready in: 00:00:00.000
2016-05-18 01:21:25,362 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,362 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB17C293498431DADC6938D5B1F365B, mem on close (mb), free: 137, total: 370, max: 672
2016-05-18 01:21:25,363 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - DA7372BB0EB745608A147A2C1C7D051E: Starting flush of map output
2016-05-18 01:21:25,363 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - DA7372BB0EB745608A147A2C1C7D051E: Span0.length = 10, perItem = 21
2016-05-18 01:21:25,364 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - DA7372BB0EB745608A147A2C1C7D051E: done sorting span=0, length=10, time=1
2016-05-18 01:21:25,364 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - DA7372BB0EB745608A147A2C1C7D051E: Heap = SpanIterator<0:9> (span=Span[16777216,210]),
2016-05-18 01:21:25,365 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - DA7372BB0EB745608A147A2C1C7D051E: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-local-dir/output/attempt_1463534485102_0001_1_00_000000_0_10089_0/file.out
2016-05-18 01:21:25,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:25 Completed running task attempt: attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,368 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0, askedToStop=false
2016-05-18 01:21:25,369 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534485102_0001_1_00_000000_0, stopRequested=false
2016-05-18 01:21:25,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534485102_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463534485359, Process_Duration=3, Process_End_Time=1463534485362, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-18 01:21:25,370 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534485102_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:25,370 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:25,371 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:25,371 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,371 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534485102_0001_00_000001
2016-05-18 01:21:25,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAB17C293498431DADC6938D5B1F365B, taskAttemptId=attempt_1463534485102_0001_1_00_000000_0, creationTime=1463534485313, allocationTime=1463534485315, startTime=1463534485317, finishTime=1463534485371, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:25,372 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534485102_0001_1_00_000000_0
2016-05-18 01:21:25,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_FINISHED]: vertexName=CAB17C293498431DADC6938D5B1F365B, taskId=task_1463534485102_0001_1_00_000000, startTime=1463534485317, finishTime=1463534485372, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463534485102_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534485359, Process_Duration=3, Process_End_Time=1463534485362, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-18 01:21:25,373 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:25,373 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_FINISHED]: vertexName=CAB17C293498431DADC6938D5B1F365B, vertexId=vertex_1463534485102_0001_1_00, initRequestedTime=1463534485294, initedTime=1463534485306, startRequestedTime=1463534485295, startedTime=1463534485307, finishTime=1463534485373, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463534485359, Process_Duration=3, Process_End_Time=1463534485362, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463534485317, firstTasksToStart=[ task_1463534485102_0001_1_00_000000 ], lastTaskFinishTime=1463534485371, lastTasksToFinish=[ task_1463534485102_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534485102_0001_1_00_000000 ], longestDurationTasks=[ task_1463534485102_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:25,375 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:25,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534485102_0001_00_000001, stoppedTime=1463534485375, exitStatus=0
2016-05-18 01:21:25,375 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] attempt: attempt_1463534485102_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 01:21:25,375 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:25,375 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DA7372BB0EB745608A147A2C1C7D051E
2016-05-18 01:21:25,376 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=CAB17C293498431DADC6938D5B1F365B, destinationVertexName=DA7372BB0EB745608A147A2C1C7D051E. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 01:21:25,376 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,375 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:25,377 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534485102_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 01:21:25,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_STARTED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, taskId=task_1463534485102_0001_1_01_000000, scheduledTime=1463534485376, launchTime=1463534485376
2016-05-18 01:21:25,377 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463534485102_0001_00_000002, launchTime=1463534485379
2016-05-18 01:21:25,379 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534485102_0001_00_000002
2016-05-18 01:21:25,380 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534485102_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,380 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463534485102_0001_1_01_000000_0] started. Is using containerId: [container_1463534485102_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 01:21:25 Starting to run new task attempt: attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0, startTime=1463534485380, containerId=container_1463534485102_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 01:21:25,380 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,380 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,381 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 01:21:25,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple, VertexName: DA7372BB0EB745608A147A2C1C7D051E, VertexParallelism: 1, TaskAttemptID:attempt_1463534485102_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB17C293498431DADC6938D5B1F365B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A468014CF374C83B41DD966B2CE37DB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 01:21:25,381 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 01:21:25,381 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,382 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534485102_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 01:21:25,382 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534485102_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:25,382 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534485102_0001_00_000001, Exiting
2016-05-18 01:21:25,382 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534485102_0001_00_000001
2016-05-18 01:21:25,382 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534485102_0001_00_000001
2016-05-18 01:21:25,383 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463534485102_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,384 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463534485102_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 01:21:25,384 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 01:21:25,386 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 01:21:25,387 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534485102_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:25,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534485102_0001_00_000001 completed successfully
2016-05-18 01:21:25,393 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA7372BB0EB745608A147A2C1C7D051E, ordinal: 1
2016-05-18 01:21:25,393 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA7372BB0EB745608A147A2C1C7D051E, mem on start (mb), free: 132, total: 370, max: 672
2016-05-18 01:21:25,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 01:21:25,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 01:21:25,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 01:21:25,400 INFO output.MROutput (MROutput.java:initialize(421)) - 4A468014CF374C83B41DD966B2CE37DB: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 01:21:25,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 01:21:25,400 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[CAB17C293498431DADC6938D5B1F365B:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [4A468014CF374C83B41DD966B2CE37DB:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 01:21:25,401 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 01:21:25,401 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 01:21:25,401 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 01:21:25,401 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[CAB17C293498431DADC6938D5B1F365B:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [4A468014CF374C83B41DD966B2CE37DB:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 01:21:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 01:21:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: CAB17C293498431DADC6938D5B1F365B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 01:21:25,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 01:21:25,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 01:21:25,403 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - CAB17C293498431DADC6938D5B1F365B: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 01:21:25,403 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - CAB17C293498431DADC6938D5B1F365B: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 01:21:25,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: CAB17C293498431DADC6938D5B1F365B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 01:21:25,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: CAB17C293498431DADC6938D5B1F365B
2016-05-18 01:21:25,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 01:21:25,406 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,407 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - CAB17C293498431DADC6938D5B1F365B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 01:21:25,409 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-05-18 01:21:25,410 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 95D73A26B5CE4EB69A2FB4723B471E22
2016-05-18 01:21:25,410 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 4A468014CF374C83B41DD966B2CE37DB
2016-05-18 01:21:25,410 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-05-18 01:21:25,410 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 95D73A26B5CE4EB69A2FB4723B471E22, for 1 inputs
2016-05-18 01:21:25,411 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463534485102_0001_1_00_000000_0_10089} to DISK_DIRECT, csize=204, dsize=200, EndTime=1463534485411, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 01:21:25,411 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : CAB17C293498431DADC6938D5B1F365B
2016-05-18 01:21:25,411 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:25,411 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: CAB17C293498431DADC6938D5B1F365B, wasInterrupted=false
2016-05-18 01:21:25,412 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 01:21:25,412 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 01:21:25,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001_wd/localmode-local-dir/output/attempt_1463534485102_0001_1_00_000000_0_10089/file.out, len=204, isLocal=true
2016-05-18 01:21:25,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 204 bytes from disk
2016-05-18 01:21:25,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 01:21:25,413 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 01:21:25,414 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-05-18 01:21:25,414 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : CAB17C293498431DADC6938D5B1F365B
2016-05-18 01:21:25,414 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA7372BB0EB745608A147A2C1C7D051E, all 1 inputs ready in: 00:00:00.003
2016-05-18 01:21:25,414 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - CAB17C293498431DADC6938D5B1F365B: Shuffle Runner thread complete
2016-05-18 01:21:25,415 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - CAB17C293498431DADC6938D5B1F365B: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 01:21:25,418 INFO output.MROutput (MROutput.java:close(526)) - 4A468014CF374C83B41DD966B2CE37DB closed
2016-05-18 01:21:25,418 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463534485102_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 01:21:25,419 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,419 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA7372BB0EB745608A147A2C1C7D051E, mem on close (mb), free: 129, total: 370, max: 672
2016-05-18 01:21:25,419 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: CAB17C293498431DADC6938D5B1F365B
2016-05-18 01:21:25,419 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - CAB17C293498431DADC6938D5B1F365B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 01:21:25,419 INFO output.MROutput (MROutput.java:close(526)) - 4A468014CF374C83B41DD966B2CE37DB closed
2016-05-18 01:21:25,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 01:21:25 Completed running task attempt: attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,420 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0, askedToStop=false
2016-05-18 01:21:25,420 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463534485102_0001_1_01_000000_0, stopRequested=false
2016-05-18 01:21:25,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463534485102_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463534485414, Process_Duration=5, Process_End_Time=1463534485419, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-05-18 01:21:25,422 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463534485102_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:25,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, taskAttemptId=attempt_1463534485102_0001_1_01_000000_0, creationTime=1463534485377, allocationTime=1463534485378, startTime=1463534485380, finishTime=1463534485422, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 01:21:25,422 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463534485102_0001_1_01_000000_0
2016-05-18 01:21:25,422 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 01:21:25,423 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 01:21:25,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:TASK_FINISHED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, taskId=task_1463534485102_0001_1_01_000000, startTime=1463534485380, finishTime=1463534485422, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463534485102_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534485414, Process_Duration=5, Process_End_Time=1463534485419, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-05-18 01:21:25,423 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 01:21:25,423 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E]
2016-05-18 01:21:25,423 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463534485102_0001_00_000002
2016-05-18 01:21:25,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:VERTEX_FINISHED]: vertexName=DA7372BB0EB745608A147A2C1C7D051E, vertexId=vertex_1463534485102_0001_1_01, initRequestedTime=1463534485295, initedTime=1463534485300, startRequestedTime=1463534485307, startedTime=1463534485308, finishTime=1463534485423, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463534485414, Process_Duration=5, Process_End_Time=1463534485419, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1463534485380, firstTasksToStart=[ task_1463534485102_0001_1_01_000000 ], lastTaskFinishTime=1463534485422, lastTasksToFinish=[ task_1463534485102_0001_1_01_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463534485102_0001_1_01_000000 ], longestDurationTasks=[ task_1463534485102_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 01:21:25,424 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 01:21:25,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463534485102_0001_00_000002, stoppedTime=1463534485425, exitStatus=0
2016-05-18 01:21:25,425 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 01:21:25,425 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:25,425 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463534485102_0001_1_00 [CAB17C293498431DADC6938D5B1F365B]
2016-05-18 01:21:25,425 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463534485102_0001_1
2016-05-18 01:21:25,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463534485102_0001_1
2016-05-18 01:21:25,425 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 4A468014CF374C83B41DD966B2CE37DB for vertex: vertex_1463534485102_0001_1_01 [DA7372BB0EB745608A147A2C1C7D051E], outputName: 4A468014CF374C83B41DD966B2CE37DB
2016-05-18 01:21:25,427 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534485102_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 01:21:25,428 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463534485102_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 01:21:25,434 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463534485102_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 01:21:25,434 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463534485102_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 01:21:25,434 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:4A468014CF374C83B41DD966B2CE37DB of vertex/vertexGroup:DA7372BB0EB745608A147A2C1C7D051E isVertexGroupOutput:false
2016-05-18 01:21:25,435 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 01:21:25,435 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463534485102_0001_00_000002, Exiting
2016-05-18 01:21:25,435 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463534485102_0001_00_000002
2016-05-18 01:21:25,435 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463534485102_0001_00_000002
2016-05-18 01:21:25,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463534485102_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 01:21:25,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463534485102_0001_00_000002 completed successfully
2016-05-18 01:21:25,438 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463534485102_0001_1, queueSize=0
2016-05-18 01:21:25,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463534485102_0001_1][Event:DAG_FINISHED]: dagId=dag_1463534485102_0001_1, startTime=1463534485294, finishTime=1463534485435, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=240, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927068970773, Process_Duration=8, Process_End_Time=2927068970781, Read_Duration=1, Tuples_Read=20, Tuples_Written=18, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-05-18 01:21:25,438 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463534485102_0001_1 finished with state: SUCCEEDED
2016-05-18 01:21:25,438 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463534485102_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 01:21:25 Completed Dag: dag_1463534485102_0001_1
2016-05-18 01:21:25,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463534485102_0001_1, dagState=SUCCEEDED
2016-05-18 01:21:25,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463534485102_0001_1_post
2016-05-18 01:21:25,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 01:21:25,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 01:21:25,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 01:21:25,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple, with id=dag_1463534485102_0001_1
2016-05-18 01:21:25,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA] test/(1/1) ...latform/testSimple/simple, with id=dag_1463534485102_0001_1
2016-05-18 01:21:25,448 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1463534485102_0001
2016-05-18 01:21:25,450 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 01:21:25,450 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 01:21:25,450 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 01:21:25,451 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 01:21:25,451 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 01:21:25,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 01:21:25,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 01:21:25,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 01:21:25,453 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 01:21:25,453 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 01:21:25,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 01:21:25,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 01:21:25,453 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 01:21:25,453 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 01:21:25,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/tmp/.staging/BF0E24BCC7144A02A03681579BD36C79/A5A769D9CFE148DFA50D6DCBEB6738FA/.tez/application_1463534485102_0001
2016-05-18 01:21:25,458 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple/_temporary
2016-05-18 01:21:25,459 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.379
2016-05-18 01:21:25,513 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.7.x-tez-0.8.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple