2016-05-18 00:43:03,152 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 00:43:03,184 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.cascade.RiffleCascadePlatformTest, with platform: hadoop2-tez
2016-05-18 00:43:03,218 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 00:43:03,782 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 00:43:04,152 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 00:43:04,228 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: AA4D0F13311644A3B3DE818702AB6F02
2016-05-18 00:43:04,292 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testProcessFlowWithChildCounters/children/first
2016-05-18 00:43:04,314 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testProcessFlowFlowListenerExceptionHandlingInStop/stopException/first
2016-05-18 00:43:04,556 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:04,558 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:04,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.060
2016-05-18 00:43:04,827 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.073
2016-05-18 00:43:04,829 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:04,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:04,831 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:04,990 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 00:43:04,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first
2016-05-18 00:43:05,215 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:05,216 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:05,304 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-05-18 00:43:05,321 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.053
2016-05-18 00:43:05,321 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:05,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first
2016-05-18 00:43:05,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second
2016-05-18 00:43:05,452 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:05,453 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:05,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 00:43:05,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-18 00:43:05,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:05,560 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second
2016-05-18 00:43:05,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third
2016-05-18 00:43:05,624 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:05,624 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:05,688 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 00:43:05,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-05-18 00:43:05,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:05,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:05,723 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third
2016-05-18 00:43:05,725 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth
2016-05-18 00:43:05,834 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 00:43:05,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] starting
2016-05-18 00:43:05,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:05,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] sink: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"]
2016-05-18 00:43:05,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] parallel execution of steps is enabled: false
2016-05-18 00:43:05,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executing total steps: 1
2016-05-18 00:43:05,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] allocating management threads: 1
2016-05-18 00:43:05,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] starting step: (1/1) ...pleRiffle/perpetual/first
2016-05-18 00:43:05,838 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [first] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:05,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tez session mode enabled: true
2016-05-18 00:43:06,028 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:06,039 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:06,039 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:06,113 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001 doesn't exist and is created
2016-05-18 00:43:06,177 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:06,177 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:06,178 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001_wd
2016-05-18 00:43:06,241 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532186038_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:06,254 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:06,255 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:06,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:06,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:06,261 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:06,261 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:06,277 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 00:43:06,277 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 00:43:06,277 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:06,283 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:06,308 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:06,310 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:06,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:06,312 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:06,313 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:06,313 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:06,314 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:06,316 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:06,316 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:06,317 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:06,324 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:06,328 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:06,342 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:06,345 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:06,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:06,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001_wd/localmode-log-dir
2016-05-18 00:43:06,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001_wd/localmode-log-dir/history.txt.appattempt_1463532186038_0001_000000, maxErrors=10
2016-05-18 00:43:06,349 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:06,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532186038_0001_000000, appSubmitTime=1463532186222, launchTime=1463532186240
2016-05-18 00:43:06,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:06,378 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 00:43:06,378 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 00:43:06,378 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:06,421 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59697
2016-05-18 00:43:06,423 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532186038
2016-05-18 00:43:06,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532186038_0001_000000, startTime=1463532186435
2016-05-18 00:43:06,436 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:06,478 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:06,493 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:06,493 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pleRiffle/perpetual/first, applicationId=application_1463532186038_0001, dagName=[DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first
2016-05-18 00:43:06,684 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first
2016-05-18 00:43:06,693 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:06,752 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532186038_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001_wd/localmode-log-dir/dag_1463532186038_0001_1.dot
2016-05-18 00:43:06,756 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001_wd/localmode-log-dir/dag_1463532186038_0001_1-tez-dag.pb.txt
2016-05-18 00:43:06,793 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532186038_0001_1
2016-05-18 00:43:06,794 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:06,794 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first
2016-05-18 00:43:06 Running Dag: dag_1463532186038_0001_1
2016-05-18 00:43:06,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532186038_0001_1, submitTime=1463532186685
2016-05-18 00:43:06,865 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001/tez-conf.pb" } size: 36915 timestamp: 1463532186000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:06,865 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,866 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,867 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:06,868 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:06,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532186038_0001_1, initTime=1463532186870
2016-05-18 00:43:06,871 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532186038_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:06,872 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:06,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:DAG_STARTED]: dagID=dag_1463532186038_0001_1, startTime=1463532186875
2016-05-18 00:43:06,875 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532186038_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:06,876 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C69B13E67D9D48229167B949D6CC950D : {0BFB38C435244C1DADB44428E26712F5={InputName=0BFB38C435244C1DADB44428E26712F5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:06,876 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0BFB38C435244C1DADB44428E26712F5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:06,877 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,887 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pleRiffle/perpetual/first, applicationId=application_1463532186038_0001, dagName=[DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first
2016-05-18 00:43:06,891 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,892 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,894 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,902 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0BFB38C435244C1DADB44428E26712F5 on vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:06,916 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:06,926 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0BFB38C435244C1DADB44428E26712F5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:06,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] submitted tez dag to app master: application_1463532186038_0001, with dag id: dag_1463532186038_0001_1
2016-05-18 00:43:06,971 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:06,972 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:06,973 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:06,974 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:06,992 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0BFB38C435244C1DADB44428E26712F5 on vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:07,004 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:07,005 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0BFB38C435244C1DADB44428E26712F5=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:07,018 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] parallelism set to 1
2016-05-18 00:43:07,019 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:07,027 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D], numAdditionalOutputs=1
2016-05-18 00:43:07,027 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=DE81A94EA130492F818826F7FC22A18D, vertex=vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:07,034 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C69B13E67D9D48229167B949D6CC950D:DE81A94EA130492F818826F7FC22A18D using oldmapred API
2016-05-18 00:43:07,035 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=DE81A94EA130492F818826F7FC22A18D, vertexName=C69B13E67D9D48229167B949D6CC950D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:07,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C69B13E67D9D48229167B949D6CC950D, vertexId=vertex_1463532186038_0001_1_00, initRequestedTime=1463532186876, initedTime=1463532187027, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:07,046 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] with distanceFromRoot: 0
2016-05-18 00:43:07,046 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:07,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532186038_0001_1_00, startRequestedTime=1463532186916, startedTime=1463532187046
2016-05-18 00:43:07,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532186038_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:07,050 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:07,051 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C69B13E67D9D48229167B949D6CC950D
2016-05-18 00:43:07,052 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:07,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:TASK_STARTED]: vertexName=C69B13E67D9D48229167B949D6CC950D, taskId=task_1463532186038_0001_1_00_000000, scheduledTime=1463532187056, launchTime=1463532187056
2016-05-18 00:43:07,083 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:07,131 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:07,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532186038_0001_00_000001, launchTime=1463532187149
2016-05-18 00:43:07,154 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532186038_0001_00_000001
2016-05-18 00:43:07,157 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532186038_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,159 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532186038_0001_1_00_000000_0] started. Is using containerId: [container_1463532186038_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:07 Starting to run new task attempt: attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,162 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C69B13E67D9D48229167B949D6CC950D, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0, startTime=1463532187157, containerId=container_1463532186038_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:07,163 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,164 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 00:43:07,165 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:07,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first, VertexName: C69B13E67D9D48229167B949D6CC950D, VertexParallelism: 1, TaskAttemptID:attempt_1463532186038_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0BFB38C435244C1DADB44428E26712F5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DE81A94EA130492F818826F7FC22A18D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:07,179 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:07,186 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,192 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532186038_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:07,193 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532186038_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:07,215 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:07,232 INFO input.MRInput (MRInput.java:initialize(440)) - 0BFB38C435244C1DADB44428E26712F5 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:07,233 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0BFB38C435244C1DADB44428E26712F5
2016-05-18 00:43:07,243 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C69B13E67D9D48229167B949D6CC950D, ordinal: 0
2016-05-18 00:43:07,246 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C69B13E67D9D48229167B949D6CC950D, mem on start (mb), free: 159, total: 281, max: 672
2016-05-18 00:43:07,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:07,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:07,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:07,247 INFO output.MROutput (MROutput.java:initialize(421)) - DE81A94EA130492F818826F7FC22A18D: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:07,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:07,248 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0BFB38C435244C1DADB44428E26712F5:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DE81A94EA130492F818826F7FC22A18D:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:07,252 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:07,257 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:07,257 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:07,258 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0BFB38C435244C1DADB44428E26712F5:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DE81A94EA130492F818826F7FC22A18D:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:07,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:07,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0BFB38C435244C1DADB44428E26712F5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:07,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0BFB38C435244C1DADB44428E26712F5
2016-05-18 00:43:07,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:07,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:07,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:07,262 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,336 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:07,337 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0BFB38C435244C1DADB44428E26712F5 initialized RecordReader from event
2016-05-18 00:43:07,350 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:07,365 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 0BFB38C435244C1DADB44428E26712F5
2016-05-18 00:43:07,365 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"], id: DE81A94EA130492F818826F7FC22A18D
2016-05-18 00:43:07,365 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"]
2016-05-18 00:43:07,369 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:07,371 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C69B13E67D9D48229167B949D6CC950D, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:43:07,380 INFO output.MROutput (MROutput.java:close(526)) - DE81A94EA130492F818826F7FC22A18D closed
2016-05-18 00:43:07,382 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532186038_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:07,384 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,384 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C69B13E67D9D48229167B949D6CC950D, mem on close (mb), free: 253, total: 281, max: 672
2016-05-18 00:43:07,385 INFO output.MROutput (MROutput.java:close(526)) - DE81A94EA130492F818826F7FC22A18D closed
2016-05-18 00:43:07,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:07 Completed running task attempt: attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,387 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:07,388 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532186038_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:07,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532186038_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532187371, Process_Duration=12, Process_End_Time=1463532187383, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:07,396 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532186038_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:07,396 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:07,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C69B13E67D9D48229167B949D6CC950D, taskAttemptId=attempt_1463532186038_0001_1_00_000000_0, creationTime=1463532187075, allocationTime=1463532187141, startTime=1463532187157, finishTime=1463532187395, timeTaken=238, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:07,397 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:07,399 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,399 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532186038_0001_00_000001
2016-05-18 00:43:07,402 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532186038_0001_1_00_000000_0
2016-05-18 00:43:07,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:TASK_FINISHED]: vertexName=C69B13E67D9D48229167B949D6CC950D, taskId=task_1463532186038_0001_1_00_000000, startTime=1463532187157, finishTime=1463532187404, timeTaken=247, status=SUCCEEDED, successfulAttemptID=attempt_1463532186038_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187371, Process_Duration=12, Process_End_Time=1463532187383, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:07,416 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:07,416 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D]
2016-05-18 00:43:07,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:VERTEX_FINISHED]: vertexName=C69B13E67D9D48229167B949D6CC950D, vertexId=vertex_1463532186038_0001_1_00, initRequestedTime=1463532186876, initedTime=1463532187027, startRequestedTime=1463532186916, startedTime=1463532187046, finishTime=1463532187417, timeTaken=371, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187371, Process_Duration=12, Process_End_Time=1463532187383, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532187157, firstTasksToStart=[ task_1463532186038_0001_1_00_000000 ], lastTaskFinishTime=1463532187395, lastTasksToFinish=[ task_1463532186038_0001_1_00_000000 ], minTaskDuration=238, maxTaskDuration=238, avgTaskDuration=238.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532186038_0001_1_00_000000 ], longestDurationTasks=[ task_1463532186038_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:07,432 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:07,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532186038_0001_00_000001, stoppedTime=1463532187434, exitStatus=0
2016-05-18 00:43:07,436 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:07,436 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:07,438 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532186038_0001_1
2016-05-18 00:43:07,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532186038_0001_1
2016-05-18 00:43:07,440 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: DE81A94EA130492F818826F7FC22A18D for vertex: vertex_1463532186038_0001_1_00 [C69B13E67D9D48229167B949D6CC950D], outputName: DE81A94EA130492F818826F7FC22A18D
2016-05-18 00:43:07,442 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532186038_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:07,445 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532186038_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:07,453 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:DE81A94EA130492F818826F7FC22A18D of vertex/vertexGroup:C69B13E67D9D48229167B949D6CC950D isVertexGroupOutput:false
2016-05-18 00:43:07,453 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:07,459 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532186038_0001_1, queueSize=0
2016-05-18 00:43:07,461 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532186038_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:07,461 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532186038_0001_00_000001: 61 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:07,461 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532186038_0001_00_000001, Exiting
2016-05-18 00:43:07,461 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532186038_0001_00_000001
2016-05-18 00:43:07,461 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532186038_0001_00_000001
2016-05-18 00:43:07,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532186038_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532186038_0001_1, startTime=1463532186872, finishTime=1463532187453, timeTaken=581, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1430, AM_GC_TIME_MILLIS=37, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187371, Process_Duration=12, Process_End_Time=1463532187383, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:07,463 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532186038_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:07,464 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532186038_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:07 Completed Dag: dag_1463532186038_0001_1
2016-05-18 00:43:07,464 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532186038_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:07,464 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532186038_0001_1_post
2016-05-18 00:43:07,465 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,467 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:07,467 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:07,465 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532186038_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:07,468 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532186038_0001_00_000001 completed successfully
2016-05-18 00:43:07,468 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first, with id=dag_1463532186038_0001_1
2016-05-18 00:43:07,471 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086] first/(1/1) ...pleRiffle/perpetual/first, with id=dag_1463532186038_0001_1
2016-05-18 00:43:07,477 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pleRiffle/perpetual/first, applicationId=application_1463532186038_0001
2016-05-18 00:43:07,483 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:07,483 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:07,483 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:07,483 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:07,483 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:07,485 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:07,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:07,490 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:07,490 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:07,491 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:07,491 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:07,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:07,494 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:07,496 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:07,508 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DDECE51D4E3941AE8537DF2C12CE92A2/E73B729956DA407A94E433DE9A0B2086/.tez/application_1463532186038_0001
2016-05-18 00:43:07,511 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first/_temporary
2016-05-18 00:43:07,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] completed in: 00:01.674
2016-05-18 00:43:07,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] starting
2016-05-18 00:43:07,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] source: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"]
2016-05-18 00:43:07,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] sink: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"]
2016-05-18 00:43:07,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] parallel execution of steps is enabled: false
2016-05-18 00:43:07,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executing total steps: 1
2016-05-18 00:43:07,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] allocating management threads: 1
2016-05-18 00:43:07,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] starting step: (1/1) ...leRiffle/perpetual/second
2016-05-18 00:43:07,517 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [second] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:07,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tez session mode enabled: true
2016-05-18 00:43:07,555 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:07,555 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:07,556 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:07,569 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001 doesn't exist and is created
2016-05-18 00:43:07,596 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:07,597 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:07,598 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001_wd
2016-05-18 00:43:07,612 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532187555_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:07,613 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:07,613 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:07,616 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:07,616 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:07,617 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:07,617 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:07,619 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:07,621 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:07,622 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:07,622 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:07,622 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:07,622 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:07,623 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:07,624 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:07,624 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:07,624 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:07,625 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:07,626 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:07,626 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:07,629 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:07,631 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:07,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:07,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001_wd/localmode-log-dir
2016-05-18 00:43:07,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001_wd/localmode-log-dir/history.txt.appattempt_1463532187555_0001_000000, maxErrors=10
2016-05-18 00:43:07,633 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:07,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532187555_0001_000000, appSubmitTime=1463532187611, launchTime=1463532187611
2016-05-18 00:43:07,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:07,647 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40016
2016-05-18 00:43:07,648 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532187555
2016-05-18 00:43:07,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532187555_0001_000000, startTime=1463532187651
2016-05-18 00:43:07,651 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:07,697 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:07,697 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:07,697 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...leRiffle/perpetual/second, applicationId=application_1463532187555_0001, dagName=[7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second
2016-05-18 00:43:07,719 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second
2016-05-18 00:43:07,719 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:07,720 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532187555_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001_wd/localmode-log-dir/dag_1463532187555_0001_1.dot
2016-05-18 00:43:07,721 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001_wd/localmode-log-dir/dag_1463532187555_0001_1-tez-dag.pb.txt
2016-05-18 00:43:07,725 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532187555_0001_1
2016-05-18 00:43:07,725 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,726 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second
2016-05-18 00:43:07 Running Dag: dag_1463532187555_0001_1
2016-05-18 00:43:07,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532187555_0001_1, submitTime=1463532187719
2016-05-18 00:43:07,737 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001/tez-conf.pb" } size: 36915 timestamp: 1463532187000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:07,737 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,737 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,738 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:07,738 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:07,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532187555_0001_1, initTime=1463532187738
2016-05-18 00:43:07,740 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187555_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:07,741 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:07,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:DAG_STARTED]: dagID=dag_1463532187555_0001_1, startTime=1463532187742
2016-05-18 00:43:07,742 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187555_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:07,742 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FDD1648C1F944A2A92FFB6C245533FB5 : {DE81A94EA130492F818826F7FC22A18D={InputName=DE81A94EA130492F818826F7FC22A18D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:07,743 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DE81A94EA130492F818826F7FC22A18D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:07,743 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,743 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,743 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,744 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,744 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:07,745 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...leRiffle/perpetual/second, applicationId=application_1463532187555_0001, dagName=[7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second
2016-05-18 00:43:07,745 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DE81A94EA130492F818826F7FC22A18D on vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,750 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DE81A94EA130492F818826F7FC22A18D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:07,771 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:07,771 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:07,771 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:07,772 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:07,772 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DE81A94EA130492F818826F7FC22A18D on vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,775 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:07,775 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DE81A94EA130492F818826F7FC22A18D=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:07,776 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] parallelism set to 1
2016-05-18 00:43:07,776 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:07,777 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5], numAdditionalOutputs=1
2016-05-18 00:43:07,777 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=DAFA0FB21EC342E79587402F08C40481, vertex=vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:07,781 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] submitted tez dag to app master: application_1463532187555_0001, with dag id: dag_1463532187555_0001_1
2016-05-18 00:43:07,783 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for FDD1648C1F944A2A92FFB6C245533FB5:DAFA0FB21EC342E79587402F08C40481 using oldmapred API
2016-05-18 00:43:07,783 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=DAFA0FB21EC342E79587402F08C40481, vertexName=FDD1648C1F944A2A92FFB6C245533FB5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:07,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, vertexId=vertex_1463532187555_0001_1_00, initRequestedTime=1463532187742, initedTime=1463532187777, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:07,786 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] with distanceFromRoot: 0
2016-05-18 00:43:07,786 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:07,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532187555_0001_1_00, startRequestedTime=1463532187744, startedTime=1463532187787
2016-05-18 00:43:07,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532187555_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:07,788 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:07,789 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FDD1648C1F944A2A92FFB6C245533FB5
2016-05-18 00:43:07,789 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:TASK_STARTED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, taskId=task_1463532187555_0001_1_00_000000, scheduledTime=1463532187791, launchTime=1463532187791
2016-05-18 00:43:07,793 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,794 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:07,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532187555_0001_00_000001, launchTime=1463532187796
2016-05-18 00:43:07,799 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532187555_0001_00_000001
2016-05-18 00:43:07,801 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532187555_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,801 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532187555_0001_1_00_000000_0] started. Is using containerId: [container_1463532187555_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:07 Starting to run new task attempt: attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0, startTime=1463532187801, containerId=container_1463532187555_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:07,802 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,802 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,802 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:07,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second, VertexName: FDD1648C1F944A2A92FFB6C245533FB5, VertexParallelism: 1, TaskAttemptID:attempt_1463532187555_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DE81A94EA130492F818826F7FC22A18D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DAFA0FB21EC342E79587402F08C40481, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:07,803 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:07,803 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,808 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:07,808 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532187555_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:07,809 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532187555_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:07,819 INFO input.MRInput (MRInput.java:initialize(440)) - DE81A94EA130492F818826F7FC22A18D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:07,819 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DE81A94EA130492F818826F7FC22A18D
2016-05-18 00:43:07,824 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDD1648C1F944A2A92FFB6C245533FB5, ordinal: 0
2016-05-18 00:43:07,824 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDD1648C1F944A2A92FFB6C245533FB5, mem on start (mb), free: 207, total: 281, max: 672
2016-05-18 00:43:07,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:07,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:07,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:07,833 INFO output.MROutput (MROutput.java:initialize(421)) - DAFA0FB21EC342E79587402F08C40481: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:07,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:07,834 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DE81A94EA130492F818826F7FC22A18D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DAFA0FB21EC342E79587402F08C40481:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:07,834 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:07,834 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:07,835 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:07,835 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DE81A94EA130492F818826F7FC22A18D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DAFA0FB21EC342E79587402F08C40481:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:07,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:07,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DE81A94EA130492F818826F7FC22A18D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:07,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:07,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:07,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DE81A94EA130492F818826F7FC22A18D
2016-05-18 00:43:07,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:07,837 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,842 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"]
2016-05-18 00:43:07,848 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"] streamed: true, id: DE81A94EA130492F818826F7FC22A18D
2016-05-18 00:43:07,848 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"], id: DAFA0FB21EC342E79587402F08C40481
2016-05-18 00:43:07,849 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"]
2016-05-18 00:43:07,849 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first"]
2016-05-18 00:43:07,851 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:07,851 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DE81A94EA130492F818826F7FC22A18D initialized RecordReader from event
2016-05-18 00:43:07,852 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDD1648C1F944A2A92FFB6C245533FB5, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:43:07,857 INFO output.MROutput (MROutput.java:close(526)) - DAFA0FB21EC342E79587402F08C40481 closed
2016-05-18 00:43:07,858 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532187555_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:07,859 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,859 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDD1648C1F944A2A92FFB6C245533FB5, mem on close (mb), free: 205, total: 281, max: 672
2016-05-18 00:43:07,860 INFO output.MROutput (MROutput.java:close(526)) - DAFA0FB21EC342E79587402F08C40481 closed
2016-05-18 00:43:07,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:07 Completed running task attempt: attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,861 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:07,861 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532187555_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:07,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532187555_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532187852, Process_Duration=7, Process_End_Time=1463532187859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:07,863 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532187555_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:07,863 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:07,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, taskAttemptId=attempt_1463532187555_0001_1_00_000000_0, creationTime=1463532187792, allocationTime=1463532187795, startTime=1463532187801, finishTime=1463532187863, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:07,863 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:07,864 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,864 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532187555_0001_1_00_000000_0
2016-05-18 00:43:07,864 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532187555_0001_00_000001
2016-05-18 00:43:07,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:TASK_FINISHED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, taskId=task_1463532187555_0001_1_00_000000, startTime=1463532187801, finishTime=1463532187864, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463532187555_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187852, Process_Duration=7, Process_End_Time=1463532187859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:07,866 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:07,866 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5]
2016-05-18 00:43:07,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:VERTEX_FINISHED]: vertexName=FDD1648C1F944A2A92FFB6C245533FB5, vertexId=vertex_1463532187555_0001_1_00, initRequestedTime=1463532187742, initedTime=1463532187777, startRequestedTime=1463532187744, startedTime=1463532187787, finishTime=1463532187866, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187852, Process_Duration=7, Process_End_Time=1463532187859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532187801, firstTasksToStart=[ task_1463532187555_0001_1_00_000000 ], lastTaskFinishTime=1463532187863, lastTasksToFinish=[ task_1463532187555_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532187555_0001_1_00_000000 ], longestDurationTasks=[ task_1463532187555_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:07,869 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:07,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532187555_0001_00_000001, stoppedTime=1463532187870, exitStatus=0
2016-05-18 00:43:07,870 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:07,871 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:07,871 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532187555_0001_1
2016-05-18 00:43:07,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532187555_0001_1
2016-05-18 00:43:07,871 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: DAFA0FB21EC342E79587402F08C40481 for vertex: vertex_1463532187555_0001_1_00 [FDD1648C1F944A2A92FFB6C245533FB5], outputName: DAFA0FB21EC342E79587402F08C40481
2016-05-18 00:43:07,871 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187555_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:07,872 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532187555_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:07,875 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532187555_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:07,875 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532187555_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:07,875 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532187555_0001_00_000001, Exiting
2016-05-18 00:43:07,875 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532187555_0001_00_000001
2016-05-18 00:43:07,876 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532187555_0001_00_000001
2016-05-18 00:43:07,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532187555_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:07,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532187555_0001_00_000001 completed successfully
2016-05-18 00:43:07,880 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:DAFA0FB21EC342E79587402F08C40481 of vertex/vertexGroup:FDD1648C1F944A2A92FFB6C245533FB5 isVertexGroupOutput:false
2016-05-18 00:43:07,880 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:07,882 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532187555_0001_1, queueSize=0
2016-05-18 00:43:07,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187555_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532187555_0001_1, startTime=1463532187741, finishTime=1463532187880, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=340, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532187852, Process_Duration=7, Process_End_Time=1463532187859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:07,885 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532187555_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:07,885 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187555_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:07 Completed Dag: dag_1463532187555_0001_1
2016-05-18 00:43:07,886 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532187555_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:07,888 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532187555_0001_1_post
2016-05-18 00:43:07,888 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:07,888 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:07,889 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:07,889 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second, with id=dag_1463532187555_0001_1
2016-05-18 00:43:07,889 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19] second/(1/1) ...leRiffle/perpetual/second, with id=dag_1463532187555_0001_1
2016-05-18 00:43:07,893 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...leRiffle/perpetual/second, applicationId=application_1463532187555_0001
2016-05-18 00:43:07,896 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:07,897 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:07,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:07,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:07,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:07,897 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:07,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:07,899 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:07,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:07,901 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:07,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:07,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:07,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:07,906 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:07,912 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/7C013DB770EC4BF3A5F3B1F9041961F0/C11F51F02FFD44DFA32183A62F89ED19/.tez/application_1463532187555_0001
2016-05-18 00:43:07,913 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second/_temporary
2016-05-18 00:43:07,914 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] completed in: 00:00.398
2016-05-18 00:43:07,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] starting
2016-05-18 00:43:07,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] source: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"]
2016-05-18 00:43:07,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] sink: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"]
2016-05-18 00:43:07,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] parallel execution of steps is enabled: false
2016-05-18 00:43:07,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executing total steps: 1
2016-05-18 00:43:07,918 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] allocating management threads: 1
2016-05-18 00:43:07,918 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] starting step: (1/1) ...pleRiffle/perpetual/third
2016-05-18 00:43:07,918 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [third] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:07,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tez session mode enabled: true
2016-05-18 00:43:07,942 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:07,943 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:07,943 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:07,953 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001 doesn't exist and is created
2016-05-18 00:43:07,971 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:07,971 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:07,972 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001_wd
2016-05-18 00:43:07,984 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532187943_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:07,984 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:07,984 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:07,986 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:07,986 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:07,986 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:07,986 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:07,988 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:07,990 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:07,990 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:07,990 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:07,990 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:07,991 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:07,991 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:07,991 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:07,993 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:07,993 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:07,993 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:07,993 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:07,994 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:07,995 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:07,996 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:07,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:07,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001_wd/localmode-log-dir
2016-05-18 00:43:07,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001_wd/localmode-log-dir/history.txt.appattempt_1463532187943_0001_000000, maxErrors=10
2016-05-18 00:43:07,997 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:07,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532187943_0001_000000, appSubmitTime=1463532187983, launchTime=1463532187983
2016-05-18 00:43:08,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:08,009 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54459
2016-05-18 00:43:08,010 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532187943
2016-05-18 00:43:08,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532187943_0001_000000, startTime=1463532188012
2016-05-18 00:43:08,012 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:08,071 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:08,072 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:08,072 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...pleRiffle/perpetual/third, applicationId=application_1463532187943_0001, dagName=[1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third
2016-05-18 00:43:08,092 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third
2016-05-18 00:43:08,092 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:08,094 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532187943_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001_wd/localmode-log-dir/dag_1463532187943_0001_1.dot
2016-05-18 00:43:08,095 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001_wd/localmode-log-dir/dag_1463532187943_0001_1-tez-dag.pb.txt
2016-05-18 00:43:08,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532187943_0001_1
2016-05-18 00:43:08,099 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,099 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third
2016-05-18 00:43:08 Running Dag: dag_1463532187943_0001_1
2016-05-18 00:43:08,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532187943_0001_1, submitTime=1463532188092
2016-05-18 00:43:08,110 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001/tez-conf.pb" } size: 36915 timestamp: 1463532187000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:08,110 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,111 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,111 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:08,111 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:08,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532187943_0001_1, initTime=1463532188111
2016-05-18 00:43:08,113 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187943_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:08,114 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:08,114 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...pleRiffle/perpetual/third, applicationId=application_1463532187943_0001, dagName=[1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third
2016-05-18 00:43:08,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:DAG_STARTED]: dagID=dag_1463532187943_0001_1, startTime=1463532188114
2016-05-18 00:43:08,114 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187943_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:08,115 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C0760BAF27EF432CADD36ADCE42E9234 : {DAFA0FB21EC342E79587402F08C40481={InputName=DAFA0FB21EC342E79587402F08C40481}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:08,115 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: DAFA0FB21EC342E79587402F08C40481, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:08,115 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,115 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,115 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,116 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,117 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:08,118 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: DAFA0FB21EC342E79587402F08C40481 on vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,120 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input DAFA0FB21EC342E79587402F08C40481 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:08,135 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:08,135 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:08,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:08,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:08,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] submitted tez dag to app master: application_1463532187943_0001, with dag id: dag_1463532187943_0001_1
2016-05-18 00:43:08,136 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: DAFA0FB21EC342E79587402F08C40481 on vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,140 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:08,141 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {DAFA0FB21EC342E79587402F08C40481=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:08,142 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] parallelism set to 1
2016-05-18 00:43:08,142 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:08,144 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234], numAdditionalOutputs=1
2016-05-18 00:43:08,144 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=EC415509506D47DCA3175B258B2AE005, vertex=vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:08,146 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for C0760BAF27EF432CADD36ADCE42E9234:EC415509506D47DCA3175B258B2AE005 using oldmapred API
2016-05-18 00:43:08,146 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=EC415509506D47DCA3175B258B2AE005, vertexName=C0760BAF27EF432CADD36ADCE42E9234, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:08,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, vertexId=vertex_1463532187943_0001_1_00, initRequestedTime=1463532188115, initedTime=1463532188144, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:08,147 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] with distanceFromRoot: 0
2016-05-18 00:43:08,147 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:08,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532187943_0001_1_00, startRequestedTime=1463532188118, startedTime=1463532188149
2016-05-18 00:43:08,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532187943_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:08,149 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:08,150 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C0760BAF27EF432CADD36ADCE42E9234
2016-05-18 00:43:08,151 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:TASK_STARTED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, taskId=task_1463532187943_0001_1_00_000000, scheduledTime=1463532188152, launchTime=1463532188152
2016-05-18 00:43:08,153 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,154 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:08,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532187943_0001_00_000001, launchTime=1463532188156
2016-05-18 00:43:08,157 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532187943_0001_00_000001
2016-05-18 00:43:08,158 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532187943_0001_1_00_000000_0] started. Is using containerId: [container_1463532187943_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:08,158 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532187943_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08 Starting to run new task attempt: attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0, startTime=1463532188158, containerId=container_1463532187943_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:08,160 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,160 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,161 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:08,162 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third, VertexName: C0760BAF27EF432CADD36ADCE42E9234, VertexParallelism: 1, TaskAttemptID:attempt_1463532187943_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DAFA0FB21EC342E79587402F08C40481, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EC415509506D47DCA3175B258B2AE005, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:08,162 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:08,163 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,164 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532187943_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:08,166 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532187943_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:08,173 INFO input.MRInput (MRInput.java:initialize(440)) - DAFA0FB21EC342E79587402F08C40481 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:08,173 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:08,174 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: DAFA0FB21EC342E79587402F08C40481
2016-05-18 00:43:08,182 INFO output.MROutput (MROutput.java:initialize(421)) - EC415509506D47DCA3175B258B2AE005: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:08,194 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C0760BAF27EF432CADD36ADCE42E9234, ordinal: 0
2016-05-18 00:43:08,194 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0760BAF27EF432CADD36ADCE42E9234, mem on start (mb), free: 168, total: 281, max: 672
2016-05-18 00:43:08,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:08,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:08,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:08,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:08,196 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DAFA0FB21EC342E79587402F08C40481:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EC415509506D47DCA3175B258B2AE005:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:08,197 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:08,197 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:08,198 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:08,198 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DAFA0FB21EC342E79587402F08C40481:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EC415509506D47DCA3175B258B2AE005:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:08,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:08,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DAFA0FB21EC342E79587402F08C40481
2016-05-18 00:43:08,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DAFA0FB21EC342E79587402F08C40481 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:08,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:08,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:08,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:08,203 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,206 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:08,207 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - DAFA0FB21EC342E79587402F08C40481 initialized RecordReader from event
2016-05-18 00:43:08,209 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"]
2016-05-18 00:43:08,211 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"] streamed: true, id: DAFA0FB21EC342E79587402F08C40481
2016-05-18 00:43:08,212 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"], id: EC415509506D47DCA3175B258B2AE005
2016-05-18 00:43:08,212 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"]
2016-05-18 00:43:08,213 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second"]
2016-05-18 00:43:08,214 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C0760BAF27EF432CADD36ADCE42E9234, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:43:08,219 INFO output.MROutput (MROutput.java:close(526)) - EC415509506D47DCA3175B258B2AE005 closed
2016-05-18 00:43:08,221 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532187943_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:08,222 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,223 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0760BAF27EF432CADD36ADCE42E9234, mem on close (mb), free: 167, total: 281, max: 672
2016-05-18 00:43:08,223 INFO output.MROutput (MROutput.java:close(526)) - EC415509506D47DCA3175B258B2AE005 closed
2016-05-18 00:43:08,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:08 Completed running task attempt: attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,225 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:08,225 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532187943_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:08,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532187943_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532188214, Process_Duration=8, Process_End_Time=1463532188222, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:08,228 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532187943_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:08,229 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:08,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, taskAttemptId=attempt_1463532187943_0001_1_00_000000_0, creationTime=1463532188153, allocationTime=1463532188155, startTime=1463532188158, finishTime=1463532188228, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:08,229 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:08,229 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,229 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532187943_0001_1_00_000000_0
2016-05-18 00:43:08,230 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532187943_0001_00_000001
2016-05-18 00:43:08,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:TASK_FINISHED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, taskId=task_1463532187943_0001_1_00_000000, startTime=1463532188158, finishTime=1463532188230, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463532187943_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188214, Process_Duration=8, Process_End_Time=1463532188222, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:08,231 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:08,231 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234]
2016-05-18 00:43:08,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:VERTEX_FINISHED]: vertexName=C0760BAF27EF432CADD36ADCE42E9234, vertexId=vertex_1463532187943_0001_1_00, initRequestedTime=1463532188115, initedTime=1463532188144, startRequestedTime=1463532188118, startedTime=1463532188149, finishTime=1463532188232, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188214, Process_Duration=8, Process_End_Time=1463532188222, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532188158, firstTasksToStart=[ task_1463532187943_0001_1_00_000000 ], lastTaskFinishTime=1463532188228, lastTasksToFinish=[ task_1463532187943_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532187943_0001_1_00_000000 ], longestDurationTasks=[ task_1463532187943_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:08,233 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:08,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532187943_0001_00_000001, stoppedTime=1463532188233, exitStatus=0
2016-05-18 00:43:08,234 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:08,234 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:08,234 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532187943_0001_1
2016-05-18 00:43:08,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532187943_0001_1
2016-05-18 00:43:08,235 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187943_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:08,235 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: EC415509506D47DCA3175B258B2AE005 for vertex: vertex_1463532187943_0001_1_00 [C0760BAF27EF432CADD36ADCE42E9234], outputName: EC415509506D47DCA3175B258B2AE005
2016-05-18 00:43:08,235 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532187943_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:08,241 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532187943_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:08,241 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532187943_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:08,241 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532187943_0001_00_000001, Exiting
2016-05-18 00:43:08,241 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532187943_0001_00_000001
2016-05-18 00:43:08,242 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532187943_0001_00_000001
2016-05-18 00:43:08,244 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532187943_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:08,244 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:EC415509506D47DCA3175B258B2AE005 of vertex/vertexGroup:C0760BAF27EF432CADD36ADCE42E9234 isVertexGroupOutput:false
2016-05-18 00:43:08,245 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:08,244 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532187943_0001_00_000001 completed successfully
2016-05-18 00:43:08,246 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532187943_0001_1, queueSize=0
2016-05-18 00:43:08,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532187943_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532187943_0001_1, startTime=1463532188114, finishTime=1463532188245, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=270, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188214, Process_Duration=8, Process_End_Time=1463532188222, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:08,247 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532187943_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:08,247 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532187943_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:08 Completed Dag: dag_1463532187943_0001_1
2016-05-18 00:43:08,248 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532187943_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:08,248 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532187943_0001_1_post
2016-05-18 00:43:08,249 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,249 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:08,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:08,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third, with id=dag_1463532187943_0001_1
2016-05-18 00:43:08,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2] third/(1/1) ...pleRiffle/perpetual/third, with id=dag_1463532187943_0001_1
2016-05-18 00:43:08,258 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...pleRiffle/perpetual/third, applicationId=application_1463532187943_0001
2016-05-18 00:43:08,261 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:08,262 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:08,262 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:08,262 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:08,262 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:08,263 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:08,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:08,263 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:08,263 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:08,263 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:08,264 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:08,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:08,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:08,267 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:08,270 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1F2891CE216D40098F30450CF258384C/E3B7146928BD42C0AB93761DE37E0AB2/.tez/application_1463532187943_0001
2016-05-18 00:43:08,270 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third/_temporary
2016-05-18 00:43:08,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] completed in: 00:00.354
2016-05-18 00:43:08,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] starting
2016-05-18 00:43:08,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] source: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"]
2016-05-18 00:43:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth"]
2016-05-18 00:43:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] parallel execution of steps is enabled: false
2016-05-18 00:43:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executing total steps: 1
2016-05-18 00:43:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] allocating management threads: 1
2016-05-18 00:43:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] starting step: (1/1) ...leRiffle/perpetual/fourth
2016-05-18 00:43:08,275 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [fourth] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:08,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tez session mode enabled: true
2016-05-18 00:43:08,299 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:08,299 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:08,299 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:08,311 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001 doesn't exist and is created
2016-05-18 00:43:08,342 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:08,342 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:08,344 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001_wd
2016-05-18 00:43:08,362 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532188299_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:08,362 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:08,363 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:08,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:08,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:08,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:08,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:08,370 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:08,373 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:08,373 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:08,373 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:08,374 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:08,375 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:08,375 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:08,375 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:08,375 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:08,377 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:08,377 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:08,377 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:08,379 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:08,381 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:08,383 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:08,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:08,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001_wd/localmode-log-dir
2016-05-18 00:43:08,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001_wd/localmode-log-dir/history.txt.appattempt_1463532188299_0001_000000, maxErrors=10
2016-05-18 00:43:08,386 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:08,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532188299_0001_000000, appSubmitTime=1463532188361, launchTime=1463532188361
2016-05-18 00:43:08,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:08,405 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57141
2016-05-18 00:43:08,406 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532188299
2016-05-18 00:43:08,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532188299_0001_000000, startTime=1463532188408
2016-05-18 00:43:08,409 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:08,442 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:08,443 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:08,443 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...leRiffle/perpetual/fourth, applicationId=application_1463532188299_0001, dagName=[4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth
2016-05-18 00:43:08,474 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth
2016-05-18 00:43:08,474 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:08,475 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532188299_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001_wd/localmode-log-dir/dag_1463532188299_0001_1.dot
2016-05-18 00:43:08,477 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001_wd/localmode-log-dir/dag_1463532188299_0001_1-tez-dag.pb.txt
2016-05-18 00:43:08,505 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532188299_0001_1
2016-05-18 00:43:08,506 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,506 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth
2016-05-18 00:43:08 Running Dag: dag_1463532188299_0001_1
2016-05-18 00:43:08,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532188299_0001_1, submitTime=1463532188474
2016-05-18 00:43:08,519 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001/tez-conf.pb" } size: 36915 timestamp: 1463532188000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:08,519 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,519 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,520 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:08,520 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:08,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532188299_0001_1, initTime=1463532188520
2016-05-18 00:43:08,521 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532188299_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:08,523 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...leRiffle/perpetual/fourth, applicationId=application_1463532188299_0001, dagName=[4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth
2016-05-18 00:43:08,523 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:08,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:DAG_STARTED]: dagID=dag_1463532188299_0001_1, startTime=1463532188523
2016-05-18 00:43:08,524 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532188299_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:08,524 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E3324D642C6A43829F5C179198238A05 : {EC415509506D47DCA3175B258B2AE005={InputName=EC415509506D47DCA3175B258B2AE005}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:08,524 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: EC415509506D47DCA3175B258B2AE005, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:08,524 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,524 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,524 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,525 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,526 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:08,527 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: EC415509506D47DCA3175B258B2AE005 on vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,532 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input EC415509506D47DCA3175B258B2AE005 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:08,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] submitted tez dag to app master: application_1463532188299_0001, with dag id: dag_1463532188299_0001_1
2016-05-18 00:43:08,553 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:08,553 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:08,553 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:08,553 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:08,558 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: EC415509506D47DCA3175B258B2AE005 on vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,562 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:08,564 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {EC415509506D47DCA3175B258B2AE005=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:08,564 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] parallelism set to 1
2016-05-18 00:43:08,565 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:08,566 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05], numAdditionalOutputs=1
2016-05-18 00:43:08,566 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3B0F58C0A4B142C0B36D9F7F061ADE79, vertex=vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:08,568 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E3324D642C6A43829F5C179198238A05:3B0F58C0A4B142C0B36D9F7F061ADE79 using oldmapred API
2016-05-18 00:43:08,569 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3B0F58C0A4B142C0B36D9F7F061ADE79, vertexName=E3324D642C6A43829F5C179198238A05, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:08,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E3324D642C6A43829F5C179198238A05, vertexId=vertex_1463532188299_0001_1_00, initRequestedTime=1463532188524, initedTime=1463532188566, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:08,571 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] with distanceFromRoot: 0
2016-05-18 00:43:08,572 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:08,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532188299_0001_1_00, startRequestedTime=1463532188526, startedTime=1463532188574
2016-05-18 00:43:08,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532188299_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:08,575 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:08,577 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E3324D642C6A43829F5C179198238A05
2016-05-18 00:43:08,577 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:TASK_STARTED]: vertexName=E3324D642C6A43829F5C179198238A05, taskId=task_1463532188299_0001_1_00_000000, scheduledTime=1463532188578, launchTime=1463532188578
2016-05-18 00:43:08,578 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,580 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:08,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532188299_0001_00_000001, launchTime=1463532188581
2016-05-18 00:43:08,582 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532188299_0001_00_000001
2016-05-18 00:43:08,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532188299_0001_1_00_000000_0] started. Is using containerId: [container_1463532188299_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:08,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E3324D642C6A43829F5C179198238A05, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0, startTime=1463532188583, containerId=container_1463532188299_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:08,584 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532188299_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08 Starting to run new task attempt: attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,585 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,586 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,586 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:08,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth, VertexName: E3324D642C6A43829F5C179198238A05, VertexParallelism: 1, TaskAttemptID:attempt_1463532188299_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EC415509506D47DCA3175B258B2AE005, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=3B0F58C0A4B142C0B36D9F7F061ADE79, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:08,587 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:08,589 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,590 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532188299_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:08,591 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532188299_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:08,598 INFO input.MRInput (MRInput.java:initialize(440)) - EC415509506D47DCA3175B258B2AE005 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:08,599 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: EC415509506D47DCA3175B258B2AE005
2016-05-18 00:43:08,600 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:08,609 INFO output.MROutput (MROutput.java:initialize(421)) - 3B0F58C0A4B142C0B36D9F7F061ADE79: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:08,614 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E3324D642C6A43829F5C179198238A05, ordinal: 0
2016-05-18 00:43:08,614 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3324D642C6A43829F5C179198238A05, mem on start (mb), free: 342, total: 386, max: 672
2016-05-18 00:43:08,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:08,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:08,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:08,615 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:08,615 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[EC415509506D47DCA3175B258B2AE005:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [3B0F58C0A4B142C0B36D9F7F061ADE79:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:08,616 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:08,616 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:08,616 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:08,617 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[EC415509506D47DCA3175B258B2AE005:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [3B0F58C0A4B142C0B36D9F7F061ADE79:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:08,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:08,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: EC415509506D47DCA3175B258B2AE005 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:08,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: EC415509506D47DCA3175B258B2AE005
2016-05-18 00:43:08,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:08,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:08,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:08,619 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,623 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"]
2016-05-18 00:43:08,624 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"] streamed: true, id: EC415509506D47DCA3175B258B2AE005
2016-05-18 00:43:08,624 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth"], id: 3B0F58C0A4B142C0B36D9F7F061ADE79
2016-05-18 00:43:08,624 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth"]
2016-05-18 00:43:08,625 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third"]
2016-05-18 00:43:08,626 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:08,626 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - EC415509506D47DCA3175B258B2AE005 initialized RecordReader from event
2016-05-18 00:43:08,627 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E3324D642C6A43829F5C179198238A05, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:43:08,631 INFO output.MROutput (MROutput.java:close(526)) - 3B0F58C0A4B142C0B36D9F7F061ADE79 closed
2016-05-18 00:43:08,632 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532188299_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:08,633 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,633 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3324D642C6A43829F5C179198238A05, mem on close (mb), free: 341, total: 386, max: 672
2016-05-18 00:43:08,634 INFO output.MROutput (MROutput.java:close(526)) - 3B0F58C0A4B142C0B36D9F7F061ADE79 closed
2016-05-18 00:43:08,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:08 Completed running task attempt: attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,635 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:08,635 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532188299_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:08,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532188299_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532188627, Process_Duration=6, Process_End_Time=1463532188633, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:08,636 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532188299_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:08,637 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:08,637 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:08,637 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,637 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532188299_0001_00_000001
2016-05-18 00:43:08,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E3324D642C6A43829F5C179198238A05, taskAttemptId=attempt_1463532188299_0001_1_00_000000_0, creationTime=1463532188578, allocationTime=1463532188581, startTime=1463532188583, finishTime=1463532188637, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:08,638 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532188299_0001_1_00_000000_0
2016-05-18 00:43:08,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:TASK_FINISHED]: vertexName=E3324D642C6A43829F5C179198238A05, taskId=task_1463532188299_0001_1_00_000000, startTime=1463532188583, finishTime=1463532188639, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463532188299_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188627, Process_Duration=6, Process_End_Time=1463532188633, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:08,640 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:08,641 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05]
2016-05-18 00:43:08,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:VERTEX_FINISHED]: vertexName=E3324D642C6A43829F5C179198238A05, vertexId=vertex_1463532188299_0001_1_00, initRequestedTime=1463532188524, initedTime=1463532188566, startRequestedTime=1463532188526, startedTime=1463532188574, finishTime=1463532188641, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188627, Process_Duration=6, Process_End_Time=1463532188633, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532188583, firstTasksToStart=[ task_1463532188299_0001_1_00_000000 ], lastTaskFinishTime=1463532188637, lastTasksToFinish=[ task_1463532188299_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532188299_0001_1_00_000000 ], longestDurationTasks=[ task_1463532188299_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:08,642 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:08,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532188299_0001_00_000001, stoppedTime=1463532188643, exitStatus=0
2016-05-18 00:43:08,644 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:08,644 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:08,645 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532188299_0001_1
2016-05-18 00:43:08,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532188299_0001_1
2016-05-18 00:43:08,646 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3B0F58C0A4B142C0B36D9F7F061ADE79 for vertex: vertex_1463532188299_0001_1_00 [E3324D642C6A43829F5C179198238A05], outputName: 3B0F58C0A4B142C0B36D9F7F061ADE79
2016-05-18 00:43:08,646 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532188299_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:08,647 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532188299_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:08,650 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532188299_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:08,650 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532188299_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:08,650 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532188299_0001_00_000001, Exiting
2016-05-18 00:43:08,650 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532188299_0001_00_000001
2016-05-18 00:43:08,650 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532188299_0001_00_000001
2016-05-18 00:43:08,651 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532188299_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:08,651 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532188299_0001_00_000001 completed successfully
2016-05-18 00:43:08,654 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3B0F58C0A4B142C0B36D9F7F061ADE79 of vertex/vertexGroup:E3324D642C6A43829F5C179198238A05 isVertexGroupOutput:false
2016-05-18 00:43:08,654 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:08,656 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532188299_0001_1, queueSize=0
2016-05-18 00:43:08,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532188299_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532188299_0001_1, startTime=1463532188523, finishTime=1463532188654, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=380, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532188627, Process_Duration=6, Process_End_Time=1463532188633, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:08,657 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532188299_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:08,657 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532188299_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:08 Completed Dag: dag_1463532188299_0001_1
2016-05-18 00:43:08,658 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532188299_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:08,658 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532188299_0001_1_post
2016-05-18 00:43:08,658 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:08,658 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:08,658 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:08,658 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth, with id=dag_1463532188299_0001_1
2016-05-18 00:43:08,658 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C] fourth/(1/1) ...leRiffle/perpetual/fourth, with id=dag_1463532188299_0001_1
2016-05-18 00:43:08,665 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...leRiffle/perpetual/fourth, applicationId=application_1463532188299_0001
2016-05-18 00:43:08,668 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:08,668 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:08,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:08,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:08,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:08,669 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:08,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:08,671 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:08,671 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:08,671 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:08,672 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:08,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:08,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:08,674 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:08,677 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/4CCE021CE1C1466B84AE59C3DE28EB42/C3CB70D69B28452BB786491BC358C25C/.tez/application_1463532188299_0001
2016-05-18 00:43:08,678 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth/_temporary
2016-05-18 00:43:08,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] completed in: 00:00.405
2016-05-18 00:43:08,724 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/third
2016-05-18 00:43:08,724 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/first
2016-05-18 00:43:08,724 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/fourth
2016-05-18 00:43:08,724 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffle/perpetual/second
2016-05-18 00:43:08,756 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:08,757 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:08,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-18 00:43:08,824 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 00:43:08,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:08,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:08,826 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:08,848 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 00:43:08,849 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first
2016-05-18 00:43:08,889 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:08,889 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:08,935 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 00:43:08,949 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 00:43:08,949 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:08,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:08,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:08,971 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first
2016-05-18 00:43:08,971 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second
2016-05-18 00:43:09,009 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:09,010 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:09,062 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:43:09,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.033
2016-05-18 00:43:09,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:09,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:09,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:09,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second
2016-05-18 00:43:09,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third
2016-05-18 00:43:09,131 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:43:09,131 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:43:09,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 00:43:09,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-18 00:43:09,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:09,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-18 00:43:09,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:43:09,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third
2016-05-18 00:43:09,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth
2016-05-18 00:43:09,256 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] starting
2016-05-18 00:43:09,258 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] parallel execution of flows is enabled: false
2016-05-18 00:43:09,258 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] executing total flows: 4
2016-05-18 00:43:09,258 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] allocating management threads: 1
2016-05-18 00:43:09,258 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] starting flow: first
2016-05-18 00:43:09,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] at least one sink is marked for delete
2016-05-18 00:43:09,262 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2016-05-18 00:43:09,265 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] starting
2016-05-18 00:43:09,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:09,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] sink: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"]
2016-05-18 00:43:09,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] parallel execution of steps is enabled: false
2016-05-18 00:43:09,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] executing total steps: 1
2016-05-18 00:43:09,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] allocating management threads: 1
2016-05-18 00:43:09,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] starting step: (1/1) ...de/perpetualcascade/first
2016-05-18 00:43:09,267 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [first] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:09,289 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] tez session mode enabled: true
2016-05-18 00:43:09,293 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:09,294 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:09,294 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:09,303 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001 doesn't exist and is created
2016-05-18 00:43:09,319 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:09,319 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:09,321 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001_wd
2016-05-18 00:43:09,332 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532189293_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:09,332 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:09,332 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:09,334 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:09,334 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:09,334 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:09,335 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:09,336 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:09,339 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:09,339 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:09,339 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:09,340 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:09,340 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:09,340 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:09,340 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:09,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:09,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:09,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:09,341 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:09,342 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:09,344 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:09,345 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:09,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:09,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001_wd/localmode-log-dir
2016-05-18 00:43:09,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001_wd/localmode-log-dir/history.txt.appattempt_1463532189293_0001_000000, maxErrors=10
2016-05-18 00:43:09,347 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:09,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532189293_0001_000000, appSubmitTime=1463532189331, launchTime=1463532189331
2016-05-18 00:43:09,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:09,359 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54281
2016-05-18 00:43:09,361 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532189293
2016-05-18 00:43:09,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532189293_0001_000000, startTime=1463532189364
2016-05-18 00:43:09,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:09,420 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:09,420 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:09,420 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...de/perpetualcascade/first, applicationId=application_1463532189293_0001, dagName=[1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first
2016-05-18 00:43:09,447 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first
2016-05-18 00:43:09,447 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:09,448 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532189293_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001_wd/localmode-log-dir/dag_1463532189293_0001_1.dot
2016-05-18 00:43:09,449 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001_wd/localmode-log-dir/dag_1463532189293_0001_1-tez-dag.pb.txt
2016-05-18 00:43:09,455 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532189293_0001_1
2016-05-18 00:43:09,455 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,455 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first
2016-05-18 00:43:09 Running Dag: dag_1463532189293_0001_1
2016-05-18 00:43:09,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532189293_0001_1, submitTime=1463532189447
2016-05-18 00:43:09,465 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001/tez-conf.pb" } size: 36915 timestamp: 1463532189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:09,465 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,465 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,466 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:09,466 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:09,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532189293_0001_1, initTime=1463532189466
2016-05-18 00:43:09,467 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189293_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:09,468 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:09,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:DAG_STARTED]: dagID=dag_1463532189293_0001_1, startTime=1463532189468
2016-05-18 00:43:09,469 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189293_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:09,469 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B6281F5365334C1ABDE9CCE91B50B5DA : {8BEED2200A224949A75C289871D71934={InputName=8BEED2200A224949A75C289871D71934}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:09,469 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 8BEED2200A224949A75C289871D71934, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:09,469 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,469 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,469 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,470 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,470 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...de/perpetualcascade/first, applicationId=application_1463532189293_0001, dagName=[1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first
2016-05-18 00:43:09,470 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:09,471 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 8BEED2200A224949A75C289871D71934 on vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,474 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 8BEED2200A224949A75C289871D71934 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:09,478 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:09,478 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:09,479 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:09,479 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:09,481 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 8BEED2200A224949A75C289871D71934 on vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,487 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:09,487 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {8BEED2200A224949A75C289871D71934=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:09,488 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] parallelism set to 1
2016-05-18 00:43:09,488 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:09,488 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA], numAdditionalOutputs=1
2016-05-18 00:43:09,488 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=944AF4E8CE274133AB388E7301DFA725, vertex=vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:09,491 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for B6281F5365334C1ABDE9CCE91B50B5DA:944AF4E8CE274133AB388E7301DFA725 using oldmapred API
2016-05-18 00:43:09,491 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=944AF4E8CE274133AB388E7301DFA725, vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:09,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, vertexId=vertex_1463532189293_0001_1_00, initRequestedTime=1463532189469, initedTime=1463532189488, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:09,493 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] with distanceFromRoot: 0
2016-05-18 00:43:09,495 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:09,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532189293_0001_1_00, startRequestedTime=1463532189471, startedTime=1463532189495
2016-05-18 00:43:09,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532189293_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:09,496 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:09,497 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B6281F5365334C1ABDE9CCE91B50B5DA
2016-05-18 00:43:09,497 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:TASK_STARTED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, taskId=task_1463532189293_0001_1_00_000000, scheduledTime=1463532189497, launchTime=1463532189497
2016-05-18 00:43:09,498 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,499 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:09,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532189293_0001_00_000001, launchTime=1463532189503
2016-05-18 00:43:09,504 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532189293_0001_00_000001
2016-05-18 00:43:09,506 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532189293_0001_1_00_000000_0] started. Is using containerId: [container_1463532189293_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:09,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0, startTime=1463532189505, containerId=container_1463532189293_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:09,507 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532189293_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09 Starting to run new task attempt: attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,508 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,509 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:09,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first, VertexName: B6281F5365334C1ABDE9CCE91B50B5DA, VertexParallelism: 1, TaskAttemptID:attempt_1463532189293_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8BEED2200A224949A75C289871D71934, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=944AF4E8CE274133AB388E7301DFA725, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:09,510 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:09,510 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,514 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532189293_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:09,514 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532189293_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:09,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] submitted tez dag to app master: application_1463532189293_0001, with dag id: dag_1463532189293_0001_1
2016-05-18 00:43:09,522 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:09,533 INFO input.MRInput (MRInput.java:initialize(440)) - 8BEED2200A224949A75C289871D71934 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:09,533 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 8BEED2200A224949A75C289871D71934
2016-05-18 00:43:09,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B6281F5365334C1ABDE9CCE91B50B5DA, ordinal: 0
2016-05-18 00:43:09,541 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6281F5365334C1ABDE9CCE91B50B5DA, mem on start (mb), free: 180, total: 386, max: 672
2016-05-18 00:43:09,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:09,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:09,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:09,543 INFO output.MROutput (MROutput.java:initialize(421)) - 944AF4E8CE274133AB388E7301DFA725: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:09,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:09,543 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[8BEED2200A224949A75C289871D71934:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [944AF4E8CE274133AB388E7301DFA725:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:09,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:09,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:09,544 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:09,544 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[8BEED2200A224949A75C289871D71934:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [944AF4E8CE274133AB388E7301DFA725:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 8BEED2200A224949A75C289871D71934
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 8BEED2200A224949A75C289871D71934 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:09,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:09,547 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,549 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:09,549 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 8BEED2200A224949A75C289871D71934 initialized RecordReader from event
2016-05-18 00:43:09,553 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:09,554 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 8BEED2200A224949A75C289871D71934
2016-05-18 00:43:09,554 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"], id: 944AF4E8CE274133AB388E7301DFA725
2016-05-18 00:43:09,554 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"]
2016-05-18 00:43:09,555 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:43:09,555 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B6281F5365334C1ABDE9CCE91B50B5DA, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:43:09,559 INFO output.MROutput (MROutput.java:close(526)) - 944AF4E8CE274133AB388E7301DFA725 closed
2016-05-18 00:43:09,560 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532189293_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:09,561 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,561 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6281F5365334C1ABDE9CCE91B50B5DA, mem on close (mb), free: 179, total: 386, max: 672
2016-05-18 00:43:09,561 INFO output.MROutput (MROutput.java:close(526)) - 944AF4E8CE274133AB388E7301DFA725 closed
2016-05-18 00:43:09,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:09 Completed running task attempt: attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,564 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:09,564 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532189293_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:09,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532189293_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532189555, Process_Duration=6, Process_End_Time=1463532189561, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:09,566 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532189293_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:09,567 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:09,567 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:09,567 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,567 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532189293_0001_00_000001
2016-05-18 00:43:09,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, taskAttemptId=attempt_1463532189293_0001_1_00_000000_0, creationTime=1463532189498, allocationTime=1463532189502, startTime=1463532189505, finishTime=1463532189566, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:09,568 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532189293_0001_1_00_000000_0
2016-05-18 00:43:09,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:TASK_FINISHED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, taskId=task_1463532189293_0001_1_00_000000, startTime=1463532189505, finishTime=1463532189569, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463532189293_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189555, Process_Duration=6, Process_End_Time=1463532189561, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:09,570 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:09,570 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA]
2016-05-18 00:43:09,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:VERTEX_FINISHED]: vertexName=B6281F5365334C1ABDE9CCE91B50B5DA, vertexId=vertex_1463532189293_0001_1_00, initRequestedTime=1463532189469, initedTime=1463532189488, startRequestedTime=1463532189471, startedTime=1463532189495, finishTime=1463532189570, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189555, Process_Duration=6, Process_End_Time=1463532189561, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532189505, firstTasksToStart=[ task_1463532189293_0001_1_00_000000 ], lastTaskFinishTime=1463532189566, lastTasksToFinish=[ task_1463532189293_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532189293_0001_1_00_000000 ], longestDurationTasks=[ task_1463532189293_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:09,571 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:09,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532189293_0001_00_000001, stoppedTime=1463532189571, exitStatus=0
2016-05-18 00:43:09,571 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:09,571 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:09,572 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532189293_0001_1
2016-05-18 00:43:09,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532189293_0001_1
2016-05-18 00:43:09,572 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 944AF4E8CE274133AB388E7301DFA725 for vertex: vertex_1463532189293_0001_1_00 [B6281F5365334C1ABDE9CCE91B50B5DA], outputName: 944AF4E8CE274133AB388E7301DFA725
2016-05-18 00:43:09,574 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189293_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:09,574 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532189293_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:09,578 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532189293_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:09,578 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532189293_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:09,578 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532189293_0001_00_000001, Exiting
2016-05-18 00:43:09,578 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532189293_0001_00_000001
2016-05-18 00:43:09,578 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532189293_0001_00_000001
2016-05-18 00:43:09,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532189293_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:09,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532189293_0001_00_000001 completed successfully
2016-05-18 00:43:09,581 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:944AF4E8CE274133AB388E7301DFA725 of vertex/vertexGroup:B6281F5365334C1ABDE9CCE91B50B5DA isVertexGroupOutput:false
2016-05-18 00:43:09,581 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:09,582 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532189293_0001_1, queueSize=0
2016-05-18 00:43:09,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189293_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532189293_0001_1, startTime=1463532189468, finishTime=1463532189581, timeTaken=113, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=240, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189555, Process_Duration=6, Process_End_Time=1463532189561, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:09,583 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532189293_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:09,583 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189293_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:09 Completed Dag: dag_1463532189293_0001_1
2016-05-18 00:43:09,584 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532189293_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:09,584 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532189293_0001_1_post
2016-05-18 00:43:09,584 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:09,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:09,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first, with id=dag_1463532189293_0001_1
2016-05-18 00:43:09,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34] first/(1/1) ...de/perpetualcascade/first, with id=dag_1463532189293_0001_1
2016-05-18 00:43:09,594 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...de/perpetualcascade/first, applicationId=application_1463532189293_0001
2016-05-18 00:43:09,597 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:09,597 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:09,597 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:09,597 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:09,598 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:09,598 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:09,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:09,599 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:09,599 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:09,599 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:09,599 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:09,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:09,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:09,604 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:09,607 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/1713C14C4EA94C33834449199E4A9FEC/B94DE0369E444FE7B936EC713C725C34/.tez/application_1463532189293_0001
2016-05-18 00:43:09,608 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first/_temporary
2016-05-18 00:43:09,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [first] completed in: 00:00.342
2016-05-18 00:43:09,609 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] completed flow: first
2016-05-18 00:43:09,609 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] starting flow: second
2016-05-18 00:43:09,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] at least one sink is marked for delete
2016-05-18 00:43:09,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2016-05-18 00:43:09,614 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] starting
2016-05-18 00:43:09,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] source: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"]
2016-05-18 00:43:09,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] sink: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"]
2016-05-18 00:43:09,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] parallel execution of steps is enabled: false
2016-05-18 00:43:09,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] executing total steps: 1
2016-05-18 00:43:09,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] allocating management threads: 1
2016-05-18 00:43:09,617 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] starting step: (1/1) ...e/perpetualcascade/second
2016-05-18 00:43:09,617 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [second] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:09,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] tez session mode enabled: true
2016-05-18 00:43:09,650 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:09,650 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:09,650 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:09,660 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001 doesn't exist and is created
2016-05-18 00:43:09,676 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:09,676 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:09,678 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001_wd
2016-05-18 00:43:09,689 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532189650_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:09,689 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:09,690 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:09,692 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:09,692 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:09,692 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:09,693 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:09,694 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:09,696 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:09,696 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:09,696 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:09,696 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:09,697 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:09,697 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:09,697 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:09,697 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:09,698 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:09,698 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:09,698 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:09,699 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:09,701 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:09,702 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:09,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:09,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001_wd/localmode-log-dir
2016-05-18 00:43:09,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001_wd/localmode-log-dir/history.txt.appattempt_1463532189650_0001_000000, maxErrors=10
2016-05-18 00:43:09,703 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:09,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532189650_0001_000000, appSubmitTime=1463532189688, launchTime=1463532189688
2016-05-18 00:43:09,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:09,715 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55183
2016-05-18 00:43:09,716 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532189650
2016-05-18 00:43:09,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532189650_0001_000000, startTime=1463532189718
2016-05-18 00:43:09,718 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:09,777 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:09,777 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:09,777 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...e/perpetualcascade/second, applicationId=application_1463532189650_0001, dagName=[E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second
2016-05-18 00:43:09,804 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second
2016-05-18 00:43:09,804 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:09,805 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532189650_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001_wd/localmode-log-dir/dag_1463532189650_0001_1.dot
2016-05-18 00:43:09,806 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001_wd/localmode-log-dir/dag_1463532189650_0001_1-tez-dag.pb.txt
2016-05-18 00:43:09,814 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532189650_0001_1
2016-05-18 00:43:09,814 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,814 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second
2016-05-18 00:43:09 Running Dag: dag_1463532189650_0001_1
2016-05-18 00:43:09,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532189650_0001_1, submitTime=1463532189804
2016-05-18 00:43:09,825 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001/tez-conf.pb" } size: 36915 timestamp: 1463532189000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:09,825 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,826 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,827 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:09,827 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:09,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532189650_0001_1, initTime=1463532189827
2016-05-18 00:43:09,828 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189650_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:09,830 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:09,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:DAG_STARTED]: dagID=dag_1463532189650_0001_1, startTime=1463532189830
2016-05-18 00:43:09,830 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...e/perpetualcascade/second, applicationId=application_1463532189650_0001, dagName=[E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second
2016-05-18 00:43:09,830 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189650_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: CF798E1810C44D59A2FE9275AFF4BA6C : {944AF4E8CE274133AB388E7301DFA725={InputName=944AF4E8CE274133AB388E7301DFA725}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 944AF4E8CE274133AB388E7301DFA725, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,831 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,832 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:09,832 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 944AF4E8CE274133AB388E7301DFA725 on vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,834 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 944AF4E8CE274133AB388E7301DFA725 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:09,849 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:09,850 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:09,850 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:09,850 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:09,852 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 944AF4E8CE274133AB388E7301DFA725 on vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,853 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:09,853 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {944AF4E8CE274133AB388E7301DFA725=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:09,854 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] parallelism set to 1
2016-05-18 00:43:09,854 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:09,854 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C], numAdditionalOutputs=1
2016-05-18 00:43:09,854 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=521776DDA73E4C9DA81F4F76AF6AF9E5, vertex=vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:09,858 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CF798E1810C44D59A2FE9275AFF4BA6C:521776DDA73E4C9DA81F4F76AF6AF9E5 using oldmapred API
2016-05-18 00:43:09,889 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=521776DDA73E4C9DA81F4F76AF6AF9E5, vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:09,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, vertexId=vertex_1463532189650_0001_1_00, initRequestedTime=1463532189831, initedTime=1463532189854, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:09,891 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] with distanceFromRoot: 0
2016-05-18 00:43:09,891 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:09,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532189650_0001_1_00, startRequestedTime=1463532189832, startedTime=1463532189891
2016-05-18 00:43:09,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532189650_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:09,891 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:09,892 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in CF798E1810C44D59A2FE9275AFF4BA6C
2016-05-18 00:43:09,892 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:TASK_STARTED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, taskId=task_1463532189650_0001_1_00_000000, scheduledTime=1463532189892, launchTime=1463532189892
2016-05-18 00:43:09,893 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,897 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:09,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] submitted tez dag to app master: application_1463532189650_0001, with dag id: dag_1463532189650_0001_1
2016-05-18 00:43:09,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532189650_0001_00_000001, launchTime=1463532189899
2016-05-18 00:43:09,913 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532189650_0001_00_000001
2016-05-18 00:43:09,915 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532189650_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532189650_0001_1_00_000000_0] started. Is using containerId: [container_1463532189650_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:09,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0, startTime=1463532189915, containerId=container_1463532189650_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:09 Starting to run new task attempt: attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,917 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,917 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:09,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second, VertexName: CF798E1810C44D59A2FE9275AFF4BA6C, VertexParallelism: 1, TaskAttemptID:attempt_1463532189650_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=944AF4E8CE274133AB388E7301DFA725, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=521776DDA73E4C9DA81F4F76AF6AF9E5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:09,918 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:09,918 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,927 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532189650_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:09,928 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532189650_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:09,927 INFO input.MRInput (MRInput.java:initialize(440)) - 944AF4E8CE274133AB388E7301DFA725 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:09,928 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 944AF4E8CE274133AB388E7301DFA725
2016-05-18 00:43:09,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:09,936 INFO output.MROutput (MROutput.java:initialize(421)) - 521776DDA73E4C9DA81F4F76AF6AF9E5: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:09,941 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CF798E1810C44D59A2FE9275AFF4BA6C, ordinal: 0
2016-05-18 00:43:09,942 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF798E1810C44D59A2FE9275AFF4BA6C, mem on start (mb), free: 347, total: 387, max: 672
2016-05-18 00:43:09,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:09,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:09,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:09,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:09,944 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[521776DDA73E4C9DA81F4F76AF6AF9E5:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [944AF4E8CE274133AB388E7301DFA725:INPUT:0:org.apache.tez.mapreduce.input.MRInput]
2016-05-18 00:43:09,944 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:09,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:09,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:09,945 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[521776DDA73E4C9DA81F4F76AF6AF9E5:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [944AF4E8CE274133AB388E7301DFA725:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0]
2016-05-18 00:43:09,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:09,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 944AF4E8CE274133AB388E7301DFA725 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:09,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 944AF4E8CE274133AB388E7301DFA725
2016-05-18 00:43:09,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:09,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:09,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:09,947 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,951 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"]
2016-05-18 00:43:09,951 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"] streamed: true, id: 944AF4E8CE274133AB388E7301DFA725
2016-05-18 00:43:09,952 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"], id: 521776DDA73E4C9DA81F4F76AF6AF9E5
2016-05-18 00:43:09,952 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"]
2016-05-18 00:43:09,952 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['ip']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first"]
2016-05-18 00:43:09,954 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:09,954 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 944AF4E8CE274133AB388E7301DFA725 initialized RecordReader from event
2016-05-18 00:43:09,955 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CF798E1810C44D59A2FE9275AFF4BA6C, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:43:09,960 INFO output.MROutput (MROutput.java:close(526)) - 521776DDA73E4C9DA81F4F76AF6AF9E5 closed
2016-05-18 00:43:09,961 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532189650_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:09,961 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,962 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF798E1810C44D59A2FE9275AFF4BA6C, mem on close (mb), free: 345, total: 387, max: 672
2016-05-18 00:43:09,962 INFO output.MROutput (MROutput.java:close(526)) - 521776DDA73E4C9DA81F4F76AF6AF9E5 closed
2016-05-18 00:43:09,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:09 Completed running task attempt: attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,963 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:09,963 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532189650_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:09,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532189650_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532189955, Process_Duration=6, Process_End_Time=1463532189961, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:09,965 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532189650_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:09,965 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:09,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, taskAttemptId=attempt_1463532189650_0001_1_00_000000_0, creationTime=1463532189893, allocationTime=1463532189898, startTime=1463532189915, finishTime=1463532189965, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:09,966 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532189650_0001_1_00_000000_0
2016-05-18 00:43:09,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:TASK_FINISHED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, taskId=task_1463532189650_0001_1_00_000000, startTime=1463532189915, finishTime=1463532189966, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463532189650_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189955, Process_Duration=6, Process_End_Time=1463532189961, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:09,967 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:09,967 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C]
2016-05-18 00:43:09,966 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:09,969 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:VERTEX_FINISHED]: vertexName=CF798E1810C44D59A2FE9275AFF4BA6C, vertexId=vertex_1463532189650_0001_1_00, initRequestedTime=1463532189831, initedTime=1463532189854, startRequestedTime=1463532189832, startedTime=1463532189891, finishTime=1463532189967, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189955, Process_Duration=6, Process_End_Time=1463532189961, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532189915, firstTasksToStart=[ task_1463532189650_0001_1_00_000000 ], lastTaskFinishTime=1463532189965, lastTasksToFinish=[ task_1463532189650_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532189650_0001_1_00_000000 ], longestDurationTasks=[ task_1463532189650_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:09,969 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:09,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532189650_0001_00_000001, stoppedTime=1463532189969, exitStatus=0
2016-05-18 00:43:09,970 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:09,970 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:09,970 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532189650_0001_1
2016-05-18 00:43:09,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532189650_0001_1
2016-05-18 00:43:09,970 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189650_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:09,971 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 521776DDA73E4C9DA81F4F76AF6AF9E5 for vertex: vertex_1463532189650_0001_1_00 [CF798E1810C44D59A2FE9275AFF4BA6C], outputName: 521776DDA73E4C9DA81F4F76AF6AF9E5
2016-05-18 00:43:09,975 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532189650_0001_00_000001
2016-05-18 00:43:09,975 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532189650_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:09,976 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532189650_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:09,976 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532189650_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:09,976 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532189650_0001_00_000001, Exiting
2016-05-18 00:43:09,976 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532189650_0001_00_000001
2016-05-18 00:43:09,976 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532189650_0001_00_000001
2016-05-18 00:43:09,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532189650_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:09,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532189650_0001_00_000001 completed successfully
2016-05-18 00:43:09,981 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:521776DDA73E4C9DA81F4F76AF6AF9E5 of vertex/vertexGroup:CF798E1810C44D59A2FE9275AFF4BA6C isVertexGroupOutput:false
2016-05-18 00:43:09,981 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:09,983 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532189650_0001_1, queueSize=0
2016-05-18 00:43:09,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532189650_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532189650_0001_1, startTime=1463532189830, finishTime=1463532189982, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=360, AM_GC_TIME_MILLIS=31, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532189955, Process_Duration=6, Process_End_Time=1463532189961, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:09,983 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532189650_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:09,983 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532189650_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:09 Completed Dag: dag_1463532189650_0001_1
2016-05-18 00:43:09,984 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532189650_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:09,984 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532189650_0001_1_post
2016-05-18 00:43:09,984 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:09,984 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:09,984 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:09,985 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second, with id=dag_1463532189650_0001_1
2016-05-18 00:43:09,985 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B] second/(1/1) ...e/perpetualcascade/second, with id=dag_1463532189650_0001_1
2016-05-18 00:43:09,994 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...e/perpetualcascade/second, applicationId=application_1463532189650_0001
2016-05-18 00:43:09,997 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:09,997 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:09,998 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:09,998 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:09,998 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:09,999 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:09,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:09,999 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:10,001 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:10,001 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,001 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:10,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:10,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,002 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:10,005 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E83C67EBDA1D440187A383CA74112E20/F24C7CBDDD6445A3ADFE0E19BF8A632B/.tez/application_1463532189650_0001
2016-05-18 00:43:10,006 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second/_temporary
2016-05-18 00:43:10,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [second] completed in: 00:00.392
2016-05-18 00:43:10,007 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] completed flow: second
2016-05-18 00:43:10,008 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] starting flow: third
2016-05-18 00:43:10,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] at least one sink is marked for delete
2016-05-18 00:43:10,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2016-05-18 00:43:10,011 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] starting
2016-05-18 00:43:10,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] source: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"]
2016-05-18 00:43:10,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] sink: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"]
2016-05-18 00:43:10,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] parallel execution of steps is enabled: false
2016-05-18 00:43:10,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] executing total steps: 1
2016-05-18 00:43:10,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] allocating management threads: 1
2016-05-18 00:43:10,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] starting step: (1/1) ...de/perpetualcascade/third
2016-05-18 00:43:10,013 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [third] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:10,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] tez session mode enabled: true
2016-05-18 00:43:10,033 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:10,034 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:10,034 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:10,042 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001 doesn't exist and is created
2016-05-18 00:43:10,058 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:10,058 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:10,059 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001_wd
2016-05-18 00:43:10,069 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532190033_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:10,069 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:10,069 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:10,071 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:10,072 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:10,072 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:10,072 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:10,073 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:10,075 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:10,075 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:10,075 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:10,075 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:10,075 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:10,075 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:10,076 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:10,076 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:10,076 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:10,076 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:10,078 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:10,078 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:10,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:10,081 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:10,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:10,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001_wd/localmode-log-dir
2016-05-18 00:43:10,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001_wd/localmode-log-dir/history.txt.appattempt_1463532190033_0001_000000, maxErrors=10
2016-05-18 00:43:10,082 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:10,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532190033_0001_000000, appSubmitTime=1463532190068, launchTime=1463532190069
2016-05-18 00:43:10,088 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:10,092 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48391
2016-05-18 00:43:10,092 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532190033
2016-05-18 00:43:10,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532190033_0001_000000, startTime=1463532190094
2016-05-18 00:43:10,094 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:10,158 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:10,159 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:10,159 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...de/perpetualcascade/third, applicationId=application_1463532190033_0001, dagName=[06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third
2016-05-18 00:43:10,177 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third
2016-05-18 00:43:10,178 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:10,179 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532190033_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001_wd/localmode-log-dir/dag_1463532190033_0001_1.dot
2016-05-18 00:43:10,180 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001_wd/localmode-log-dir/dag_1463532190033_0001_1-tez-dag.pb.txt
2016-05-18 00:43:10,184 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532190033_0001_1
2016-05-18 00:43:10,185 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,185 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third
2016-05-18 00:43:10 Running Dag: dag_1463532190033_0001_1
2016-05-18 00:43:10,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532190033_0001_1, submitTime=1463532190178
2016-05-18 00:43:10,197 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001/tez-conf.pb" } size: 36915 timestamp: 1463532190000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:10,197 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,197 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,197 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:10,198 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:10,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532190033_0001_1, initTime=1463532190198
2016-05-18 00:43:10,199 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190033_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:10,199 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:10,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:DAG_STARTED]: dagID=dag_1463532190033_0001_1, startTime=1463532190200
2016-05-18 00:43:10,200 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190033_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:10,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DEB9480FA09349AEA9ADBC0BB981EAF9 : {521776DDA73E4C9DA81F4F76AF6AF9E5={InputName=521776DDA73E4C9DA81F4F76AF6AF9E5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:10,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 521776DDA73E4C9DA81F4F76AF6AF9E5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:10,200 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,201 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,201 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,201 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,202 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...de/perpetualcascade/third, applicationId=application_1463532190033_0001, dagName=[06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third
2016-05-18 00:43:10,202 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:10,202 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 521776DDA73E4C9DA81F4F76AF6AF9E5 on vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,204 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 521776DDA73E4C9DA81F4F76AF6AF9E5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:10,219 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:10,219 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:10,219 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:10,220 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:10,220 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 521776DDA73E4C9DA81F4F76AF6AF9E5 on vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,221 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:10,222 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {521776DDA73E4C9DA81F4F76AF6AF9E5=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:10,223 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] parallelism set to 1
2016-05-18 00:43:10,223 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:10,225 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9], numAdditionalOutputs=1
2016-05-18 00:43:10,225 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=4DEE366AC76D41F3A57B73B57A2DE186, vertex=vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:10,227 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DEB9480FA09349AEA9ADBC0BB981EAF9:4DEE366AC76D41F3A57B73B57A2DE186 using oldmapred API
2016-05-18 00:43:10,227 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=4DEE366AC76D41F3A57B73B57A2DE186, vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:10,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, vertexId=vertex_1463532190033_0001_1_00, initRequestedTime=1463532190200, initedTime=1463532190225, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:10,228 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] with distanceFromRoot: 0
2016-05-18 00:43:10,228 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:10,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532190033_0001_1_00, startRequestedTime=1463532190202, startedTime=1463532190228
2016-05-18 00:43:10,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532190033_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:10,230 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:10,231 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DEB9480FA09349AEA9ADBC0BB981EAF9
2016-05-18 00:43:10,232 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:TASK_STARTED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, taskId=task_1463532190033_0001_1_00_000000, scheduledTime=1463532190232, launchTime=1463532190232
2016-05-18 00:43:10,233 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,236 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:10,238 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] submitted tez dag to app master: application_1463532190033_0001, with dag id: dag_1463532190033_0001_1
2016-05-18 00:43:10,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532190033_0001_00_000001, launchTime=1463532190238
2016-05-18 00:43:10,239 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532190033_0001_00_000001
2016-05-18 00:43:10,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532190033_0001_1_00_000000_0] started. Is using containerId: [container_1463532190033_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:10,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0, startTime=1463532190240, containerId=container_1463532190033_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:10,243 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532190033_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10 Starting to run new task attempt: attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,244 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,244 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,244 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:10,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third, VertexName: DEB9480FA09349AEA9ADBC0BB981EAF9, VertexParallelism: 1, TaskAttemptID:attempt_1463532190033_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=521776DDA73E4C9DA81F4F76AF6AF9E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4DEE366AC76D41F3A57B73B57A2DE186, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:10,244 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:10,245 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,250 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532190033_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:10,251 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532190033_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:10,251 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:10,252 INFO input.MRInput (MRInput.java:initialize(440)) - 521776DDA73E4C9DA81F4F76AF6AF9E5 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:10,252 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 521776DDA73E4C9DA81F4F76AF6AF9E5
2016-05-18 00:43:10,259 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DEB9480FA09349AEA9ADBC0BB981EAF9, ordinal: 0
2016-05-18 00:43:10,259 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEB9480FA09349AEA9ADBC0BB981EAF9, mem on start (mb), free: 307, total: 387, max: 672
2016-05-18 00:43:10,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:10,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:10,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:10,261 INFO output.MROutput (MROutput.java:initialize(421)) - 4DEE366AC76D41F3A57B73B57A2DE186: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:10,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:10,262 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[521776DDA73E4C9DA81F4F76AF6AF9E5:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [4DEE366AC76D41F3A57B73B57A2DE186:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:10,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:10,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:10,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:10,262 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[521776DDA73E4C9DA81F4F76AF6AF9E5:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [4DEE366AC76D41F3A57B73B57A2DE186:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:10,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:10,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 521776DDA73E4C9DA81F4F76AF6AF9E5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:10,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 521776DDA73E4C9DA81F4F76AF6AF9E5
2016-05-18 00:43:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:10,265 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,267 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:10,267 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 521776DDA73E4C9DA81F4F76AF6AF9E5 initialized RecordReader from event
2016-05-18 00:43:10,269 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"]
2016-05-18 00:43:10,269 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"] streamed: true, id: 521776DDA73E4C9DA81F4F76AF6AF9E5
2016-05-18 00:43:10,269 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"], id: 4DEE366AC76D41F3A57B73B57A2DE186
2016-05-18 00:43:10,270 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"]
2016-05-18 00:43:10,270 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['first', 'second', 'third', 'fourth']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second"]
2016-05-18 00:43:10,270 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DEB9480FA09349AEA9ADBC0BB981EAF9, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:43:10,276 INFO output.MROutput (MROutput.java:close(526)) - 4DEE366AC76D41F3A57B73B57A2DE186 closed
2016-05-18 00:43:10,276 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532190033_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:10,278 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEB9480FA09349AEA9ADBC0BB981EAF9, mem on close (mb), free: 305, total: 387, max: 672
2016-05-18 00:43:10,278 INFO output.MROutput (MROutput.java:close(526)) - 4DEE366AC76D41F3A57B73B57A2DE186 closed
2016-05-18 00:43:10,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:10 Completed running task attempt: attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,279 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:10,280 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532190033_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:10,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532190033_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532190270, Process_Duration=7, Process_End_Time=1463532190277, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:10,281 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532190033_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:10,282 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:10,282 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:10,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, taskAttemptId=attempt_1463532190033_0001_1_00_000000_0, creationTime=1463532190233, allocationTime=1463532190237, startTime=1463532190240, finishTime=1463532190281, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:10,283 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532190033_0001_1_00_000000_0
2016-05-18 00:43:10,284 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:TASK_FINISHED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, taskId=task_1463532190033_0001_1_00_000000, startTime=1463532190240, finishTime=1463532190283, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463532190033_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190270, Process_Duration=7, Process_End_Time=1463532190277, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:10,285 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:10,285 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9]
2016-05-18 00:43:10,285 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532190033_0001_00_000001
2016-05-18 00:43:10,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:VERTEX_FINISHED]: vertexName=DEB9480FA09349AEA9ADBC0BB981EAF9, vertexId=vertex_1463532190033_0001_1_00, initRequestedTime=1463532190200, initedTime=1463532190225, startRequestedTime=1463532190202, startedTime=1463532190228, finishTime=1463532190285, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190270, Process_Duration=7, Process_End_Time=1463532190277, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532190240, firstTasksToStart=[ task_1463532190033_0001_1_00_000000 ], lastTaskFinishTime=1463532190281, lastTasksToFinish=[ task_1463532190033_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532190033_0001_1_00_000000 ], longestDurationTasks=[ task_1463532190033_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:10,286 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:10,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532190033_0001_00_000001, stoppedTime=1463532190287, exitStatus=0
2016-05-18 00:43:10,287 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:10,287 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:10,287 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532190033_0001_1
2016-05-18 00:43:10,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532190033_0001_1
2016-05-18 00:43:10,288 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 4DEE366AC76D41F3A57B73B57A2DE186 for vertex: vertex_1463532190033_0001_1_00 [DEB9480FA09349AEA9ADBC0BB981EAF9], outputName: 4DEE366AC76D41F3A57B73B57A2DE186
2016-05-18 00:43:10,290 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190033_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:10,290 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532190033_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:10,295 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:4DEE366AC76D41F3A57B73B57A2DE186 of vertex/vertexGroup:DEB9480FA09349AEA9ADBC0BB981EAF9 isVertexGroupOutput:false
2016-05-18 00:43:10,295 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:10,296 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532190033_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:10,296 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532190033_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:10,296 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532190033_0001_00_000001, Exiting
2016-05-18 00:43:10,296 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532190033_0001_00_000001
2016-05-18 00:43:10,296 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532190033_0001_00_000001
2016-05-18 00:43:10,297 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532190033_0001_1, queueSize=0
2016-05-18 00:43:10,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190033_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532190033_0001_1, startTime=1463532190199, finishTime=1463532190296, timeTaken=97, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=190, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190270, Process_Duration=7, Process_End_Time=1463532190277, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:10,298 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532190033_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:10,298 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190033_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:10 Completed Dag: dag_1463532190033_0001_1
2016-05-18 00:43:10,298 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532190033_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:10,299 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532190033_0001_1_post
2016-05-18 00:43:10,299 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,299 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:10,299 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:10,299 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third, with id=dag_1463532190033_0001_1
2016-05-18 00:43:10,299 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532190033_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:10,299 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4] third/(1/1) ...de/perpetualcascade/third, with id=dag_1463532190033_0001_1
2016-05-18 00:43:10,299 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532190033_0001_00_000001 completed successfully
2016-05-18 00:43:10,307 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...de/perpetualcascade/third, applicationId=application_1463532190033_0001
2016-05-18 00:43:10,311 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:10,311 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:10,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:10,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:10,311 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:10,312 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:10,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:10,312 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:10,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:10,313 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:10,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:10,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,314 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:10,317 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/06A33AAD2F914F1586EA75542055228C/EE59DA7E66954DD8906EBAC34F8B32A4/.tez/application_1463532190033_0001
2016-05-18 00:43:10,318 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third/_temporary
2016-05-18 00:43:10,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [third] completed in: 00:00.307
2016-05-18 00:43:10,319 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] completed flow: third
2016-05-18 00:43:10,320 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] starting flow: fourth
2016-05-18 00:43:10,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] at least one sink is marked for delete
2016-05-18 00:43:10,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2016-05-18 00:43:10,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] starting
2016-05-18 00:43:10,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] source: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"]
2016-05-18 00:43:10,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth"]
2016-05-18 00:43:10,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] parallel execution of steps is enabled: false
2016-05-18 00:43:10,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] executing total steps: 1
2016-05-18 00:43:10,325 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] allocating management threads: 1
2016-05-18 00:43:10,326 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] starting step: (1/1) ...e/perpetualcascade/fourth
2016-05-18 00:43:10,326 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [fourth] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:43:10,349 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] tez session mode enabled: true
2016-05-18 00:43:10,353 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:10,353 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:43:10,353 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:43:10,362 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001 doesn't exist and is created
2016-05-18 00:43:10,377 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:43:10,377 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:43:10,378 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001_wd
2016-05-18 00:43:10,389 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463532190353_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:43:10,389 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:10,389 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:43:10,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:43:10,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:43:10,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:43:10,392 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:43:10,393 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:43:10,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:43:10,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:43:10,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:43:10,396 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:43:10,396 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:43:10,396 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:43:10,396 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:43:10,397 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:43:10,397 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:10,397 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:43:10,398 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:43:10,398 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:43:10,400 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:43:10,401 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:43:10,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:43:10,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001_wd/localmode-log-dir
2016-05-18 00:43:10,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001_wd/localmode-log-dir/history.txt.appattempt_1463532190353_0001_000000, maxErrors=10
2016-05-18 00:43:10,403 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:43:10,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463532190353_0001_000000, appSubmitTime=1463532190388, launchTime=1463532190388
2016-05-18 00:43:10,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:43:10,413 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47741
2016-05-18 00:43:10,413 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463532190353
2016-05-18 00:43:10,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463532190353_0001_000000, startTime=1463532190415
2016-05-18 00:43:10,415 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:43:10,477 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:43:10,477 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:43:10,478 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...e/perpetualcascade/fourth, applicationId=application_1463532190353_0001, dagName=[D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth
2016-05-18 00:43:10,497 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth
2016-05-18 00:43:10,497 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:43:10,498 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463532190353_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001_wd/localmode-log-dir/dag_1463532190353_0001_1.dot
2016-05-18 00:43:10,499 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001_wd/localmode-log-dir/dag_1463532190353_0001_1-tez-dag.pb.txt
2016-05-18 00:43:10,508 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532190353_0001_1
2016-05-18 00:43:10,508 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,508 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth
2016-05-18 00:43:10 Running Dag: dag_1463532190353_0001_1
2016-05-18 00:43:10,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463532190353_0001_1, submitTime=1463532190497
2016-05-18 00:43:10,518 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001/tez-conf.pb" } size: 36915 timestamp: 1463532190000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:43:10,518 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,518 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,518 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:43:10,518 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:43:10,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463532190353_0001_1, initTime=1463532190519
2016-05-18 00:43:10,519 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190353_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:43:10,520 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:43:10,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:DAG_STARTED]: dagID=dag_1463532190353_0001_1, startTime=1463532190520
2016-05-18 00:43:10,520 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190353_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:43:10,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F2B4B653440F425596892EE88FDCA7C7 : {4DEE366AC76D41F3A57B73B57A2DE186={InputName=4DEE366AC76D41F3A57B73B57A2DE186}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:43:10,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 4DEE366AC76D41F3A57B73B57A2DE186, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:43:10,520 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,521 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,521 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,521 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,521 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:43:10,521 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 4DEE366AC76D41F3A57B73B57A2DE186 on vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,522 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...e/perpetualcascade/fourth, applicationId=application_1463532190353_0001, dagName=[D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth
2016-05-18 00:43:10,524 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 4DEE366AC76D41F3A57B73B57A2DE186 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:43:10,536 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:43:10,537 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:43:10,537 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:43:10,537 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:43:10,539 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 4DEE366AC76D41F3A57B73B57A2DE186 on vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,539 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:10,540 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] submitted tez dag to app master: application_1463532190353_0001, with dag id: dag_1463532190353_0001_1
2016-05-18 00:43:10,540 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {4DEE366AC76D41F3A57B73B57A2DE186=forAllWorkUnits=true, update=[1]}
2016-05-18 00:43:10,541 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] parallelism set to 1
2016-05-18 00:43:10,541 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:43:10,542 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7], numAdditionalOutputs=1
2016-05-18 00:43:10,543 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CCA70275DC4248DB8662A97258AF6974, vertex=vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:43:10,545 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F2B4B653440F425596892EE88FDCA7C7:CCA70275DC4248DB8662A97258AF6974 using oldmapred API
2016-05-18 00:43:10,545 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CCA70275DC4248DB8662A97258AF6974, vertexName=F2B4B653440F425596892EE88FDCA7C7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:43:10,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, vertexId=vertex_1463532190353_0001_1_00, initRequestedTime=1463532190520, initedTime=1463532190542, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:43:10,546 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] with distanceFromRoot: 0
2016-05-18 00:43:10,546 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:43:10,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463532190353_0001_1_00, startRequestedTime=1463532190522, startedTime=1463532190547
2016-05-18 00:43:10,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463532190353_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:43:10,547 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:43:10,548 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F2B4B653440F425596892EE88FDCA7C7
2016-05-18 00:43:10,548 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:TASK_STARTED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, taskId=task_1463532190353_0001_1_00_000000, scheduledTime=1463532190548, launchTime=1463532190548
2016-05-18 00:43:10,550 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,550 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:43:10,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463532190353_0001_00_000001, launchTime=1463532190552
2016-05-18 00:43:10,553 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532190353_0001_00_000001
2016-05-18 00:43:10,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463532190353_0001_1_00_000000_0] started. Is using containerId: [container_1463532190353_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:43:10,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0, startTime=1463532190554, containerId=container_1463532190353_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:43:10,554 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532190353_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10 Starting to run new task attempt: attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,555 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,555 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,556 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:43:10,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth, VertexName: F2B4B653440F425596892EE88FDCA7C7, VertexParallelism: 1, TaskAttemptID:attempt_1463532190353_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4DEE366AC76D41F3A57B73B57A2DE186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CCA70275DC4248DB8662A97258AF6974, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:43:10,556 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:43:10,557 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,558 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463532190353_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:43:10,558 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463532190353_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:43:10,560 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:43:10,561 INFO input.MRInput (MRInput.java:initialize(440)) - 4DEE366AC76D41F3A57B73B57A2DE186 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:43:10,562 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 4DEE366AC76D41F3A57B73B57A2DE186
2016-05-18 00:43:10,569 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2B4B653440F425596892EE88FDCA7C7, ordinal: 0
2016-05-18 00:43:10,569 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2B4B653440F425596892EE88FDCA7C7, mem on start (mb), free: 268, total: 387, max: 672
2016-05-18 00:43:10,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:43:10,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:43:10,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:43:10,571 INFO output.MROutput (MROutput.java:initialize(421)) - CCA70275DC4248DB8662A97258AF6974: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:43:10,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:43:10,571 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[4DEE366AC76D41F3A57B73B57A2DE186:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CCA70275DC4248DB8662A97258AF6974:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:43:10,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:43:10,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:43:10,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-18 00:43:10,572 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[4DEE366AC76D41F3A57B73B57A2DE186:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CCA70275DC4248DB8662A97258AF6974:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 4DEE366AC76D41F3A57B73B57A2DE186 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 4DEE366AC76D41F3A57B73B57A2DE186
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:43:10,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:43:10,573 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,576 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"]
2016-05-18 00:43:10,576 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"] streamed: true, id: 4DEE366AC76D41F3A57B73B57A2DE186
2016-05-18 00:43:10,576 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth"], id: CCA70275DC4248DB8662A97258AF6974
2016-05-18 00:43:10,577 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth"]
2016-05-18 00:43:10,577 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[['mangled']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third"]
2016-05-18 00:43:10,577 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third/part-v000-o000-00000:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third/part-v000-o000-00000:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:43:10,577 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 4DEE366AC76D41F3A57B73B57A2DE186 initialized RecordReader from event
2016-05-18 00:43:10,578 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2B4B653440F425596892EE88FDCA7C7, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:43:10,581 INFO output.MROutput (MROutput.java:close(526)) - CCA70275DC4248DB8662A97258AF6974 closed
2016-05-18 00:43:10,581 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463532190353_0001_1_00_000000_0 given a go for committing the task output.
2016-05-18 00:43:10,582 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,582 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2B4B653440F425596892EE88FDCA7C7, mem on close (mb), free: 266, total: 387, max: 672
2016-05-18 00:43:10,582 INFO output.MROutput (MROutput.java:close(526)) - CCA70275DC4248DB8662A97258AF6974 closed
2016-05-18 00:43:10,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:43:10 Completed running task attempt: attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,583 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:43:10,583 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463532190353_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:43:10,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463532190353_0001_1_00_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463532190578, Process_Duration=4, Process_End_Time=1463532190582, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-18 00:43:10,585 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463532190353_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:10,585 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:43:10,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, taskAttemptId=attempt_1463532190353_0001_1_00_000000_0, creationTime=1463532190549, allocationTime=1463532190552, startTime=1463532190554, finishTime=1463532190585, timeTaken=31, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:43:10,585 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:43:10,585 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,585 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463532190353_0001_1_00_000000_0
2016-05-18 00:43:10,585 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463532190353_0001_00_000001
2016-05-18 00:43:10,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:TASK_FINISHED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, taskId=task_1463532190353_0001_1_00_000000, startTime=1463532190554, finishTime=1463532190585, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1463532190353_0001_1_00_000000_0, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190578, Process_Duration=4, Process_End_Time=1463532190582, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:10,588 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:43:10,588 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7]
2016-05-18 00:43:10,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:VERTEX_FINISHED]: vertexName=F2B4B653440F425596892EE88FDCA7C7, vertexId=vertex_1463532190353_0001_1_00, initRequestedTime=1463532190520, initedTime=1463532190542, startRequestedTime=1463532190522, startedTime=1463532190547, finishTime=1463532190588, timeTaken=41, status=SUCCEEDED, diagnostics=, counters=Counters: 12, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190578, Process_Duration=4, Process_End_Time=1463532190582, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463532190554, firstTasksToStart=[ task_1463532190353_0001_1_00_000000 ], lastTaskFinishTime=1463532190585, lastTasksToFinish=[ task_1463532190353_0001_1_00_000000 ], minTaskDuration=31, maxTaskDuration=31, avgTaskDuration=31.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463532190353_0001_1_00_000000 ], longestDurationTasks=[ task_1463532190353_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:43:10,590 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:43:10,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463532190353_0001_00_000001, stoppedTime=1463532190590, exitStatus=0
2016-05-18 00:43:10,591 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-18 00:43:10,591 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:10,591 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463532190353_0001_1
2016-05-18 00:43:10,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463532190353_0001_1
2016-05-18 00:43:10,592 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CCA70275DC4248DB8662A97258AF6974 for vertex: vertex_1463532190353_0001_1_00 [F2B4B653440F425596892EE88FDCA7C7], outputName: CCA70275DC4248DB8662A97258AF6974
2016-05-18 00:43:10,592 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190353_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:43:10,592 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463532190353_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:43:10,596 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463532190353_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:43:10,596 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463532190353_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:43:10,597 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463532190353_0001_00_000001, Exiting
2016-05-18 00:43:10,597 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463532190353_0001_00_000001
2016-05-18 00:43:10,597 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463532190353_0001_00_000001
2016-05-18 00:43:10,598 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463532190353_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:43:10,598 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463532190353_0001_00_000001 completed successfully
2016-05-18 00:43:10,599 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CCA70275DC4248DB8662A97258AF6974 of vertex/vertexGroup:F2B4B653440F425596892EE88FDCA7C7 isVertexGroupOutput:false
2016-05-18 00:43:10,599 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null
2016-05-18 00:43:10,602 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463532190353_0001_1, queueSize=0
2016-05-18 00:43:10,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463532190353_0001_1][Event:DAG_FINISHED]: dagId=dag_1463532190353_0001_1, startTime=1463532190520, finishTime=1463532190600, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 16, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=120, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463532190578, Process_Duration=4, Process_End_Time=1463532190582, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-18 00:43:10,603 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463532190353_0001_1 finished with state: SUCCEEDED
2016-05-18 00:43:10,603 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463532190353_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:43:10 Completed Dag: dag_1463532190353_0001_1
2016-05-18 00:43:10,604 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463532190353_0001_1, dagState=SUCCEEDED
2016-05-18 00:43:10,604 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463532190353_0001_1_post
2016-05-18 00:43:10,605 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:43:10,605 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:43:10,605 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:43:10,605 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth, with id=dag_1463532190353_0001_1
2016-05-18 00:43:10,606 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C] fourth/(1/1) ...e/perpetualcascade/fourth, with id=dag_1463532190353_0001_1
2016-05-18 00:43:10,610 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...e/perpetualcascade/fourth, applicationId=application_1463532190353_0001
2016-05-18 00:43:10,614 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:43:10,614 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:43:10,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:43:10,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:43:10,614 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:43:10,614 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:43:10,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:43:10,615 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:43:10,616 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:43:10,616 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,616 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:43:10,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:43:10,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:43:10,617 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:43:10,620 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3C318F9301D4EC99E4BB829173E79C4/EC3DEC953CF84DC99BFECC56D482468C/.tez/application_1463532190353_0001
2016-05-18 00:43:10,620 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth/_temporary
2016-05-18 00:43:10,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [fourth] completed in: 00:00.296
2016-05-18 00:43:10,621 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [fourth+second+first+third] completed flow: fourth
2016-05-18 00:43:10,659 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/third
2016-05-18 00:43:10,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/fourth
2016-05-18 00:43:10,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/second
2016-05-18 00:43:10,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testSimpleRiffleCascade/perpetualcascade/first
2016-05-18 00:43:10,667 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testProcessFlowWithCounters/counter/first
2016-05-18 00:43:10,672 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testProcessFlowFlowListenerExceptionHandlingInComplete/completeException/first
2016-05-18 00:43:10,677 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/rifflecascadeplatform/testProcessFlowFlowListenerExceptionHandlingInStart/startException/first