2016-05-18 00:38:56,236 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-18 00:38:56,269 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.SortedValuesPlatformTest, with platform: hadoop2-tez
2016-05-18 00:38:56,311 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-18 00:38:56,834 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-18 00:38:57,157 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-18 00:38:57,383 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:38:57,384 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:38:57,472 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: D7B12C2203884EF1956C808D38336033
2016-05-18 00:38:57,725 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.138
2016-05-18 00:38:57,770 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.183
2016-05-18 00:38:57,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 00:38:57,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 00:38:57,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:38:57,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 00:38:58,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt
2016-05-18 00:38:58,134 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-05-18 00:38:58,437 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-18 00:38:58,439 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] starting
2016-05-18 00:38:58,439 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:38:58,440 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:38:58,440 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-05-18 00:38:58,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] parallel execution of steps is enabled: false
2016-05-18 00:38:58,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executing total steps: 1
2016-05-18 00:38:58,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] allocating management threads: 1
2016-05-18 00:38:58,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] starting step: (1/1) ...ed/cogroupcomparereversed
2016-05-18 00:38:58,443 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:38:58,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tez session mode enabled: true
2016-05-18 00:38:58,641 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:38:58,648 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:38:58,649 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:38:58,701 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001 doesn't exist and is created
2016-05-18 00:38:58,769 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:38:58,769 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:38:58,770 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd
2016-05-18 00:38:58,831 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531938647_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:38:58,846 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:38:58,847 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:38:58,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:38:58,854 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:38:58,854 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:38:58,854 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:38:58,870 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-18 00:38:58,870 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-18 00:38:58,870 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:38:58,880 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:38:58,905 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:38:58,906 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:38:58,908 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:38:58,909 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:38:58,909 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:38:58,910 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:38:58,911 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:38:58,912 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:38:58,913 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:38:58,913 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:38:58,922 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:38:58,926 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:38:58,946 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:38:58,949 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:38:58,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:38:58,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-log-dir
2016-05-18 00:38:58,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-log-dir/history.txt.appattempt_1463531938647_0001_000000, maxErrors=10
2016-05-18 00:38:58,954 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:38:58,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531938647_0001_000000, appSubmitTime=1463531938810, launchTime=1463531938829
2016-05-18 00:38:58,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:38:58,970 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-18 00:38:58,971 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-18 00:38:58,971 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:38:59,021 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35234
2016-05-18 00:38:59,023 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531938647
2016-05-18 00:38:59,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531938647_0001_000000, startTime=1463531939042
2016-05-18 00:38:59,042 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:38:59,071 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:38:59,088 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:38:59,088 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1463531938647_0001, dagName=[EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-05-18 00:38:59,322 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-05-18 00:38:59,332 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:38:59,398 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531938647_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-log-dir/dag_1463531938647_0001_1.dot
2016-05-18 00:38:59,403 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-log-dir/dag_1463531938647_0001_1-tez-dag.pb.txt
2016-05-18 00:38:59,459 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531938647_0001_1
2016-05-18 00:38:59,459 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:38:59,459 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-05-18 00:38:59 Running Dag: dag_1463531938647_0001_1
2016-05-18 00:38:59,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531938647_0001_1, submitTime=1463531939322
2016-05-18 00:38:59,551 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001/tez-conf.pb" } size: 36903 timestamp: 1463531938000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:38:59,551 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,552 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:38:59,554 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001/tez-conf.pb" } size: 36903 timestamp: 1463531938000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:38:59,554 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,555 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:38:59,558 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001/tez-conf.pb" } size: 36903 timestamp: 1463531938000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:38:59,558 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:38:59,560 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:38:59,569 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:38:59,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531938647_0001_1, initTime=1463531939571
2016-05-18 00:38:59,573 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531938647_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:38:59,575 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:38:59,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:DAG_STARTED]: dagID=dag_1463531938647_0001_1, startTime=1463531939577
2016-05-18 00:38:59,577 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531938647_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:38:59,578 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D3E636A48C8A4BA5B292FCD269073158 : {74131C45C4A74C10BEBD9906E1B1F42C={InputName=74131C45C4A74C10BEBD9906E1B1F42C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:38:59,578 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 74131C45C4A74C10BEBD9906E1B1F42C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:38:59,578 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,588 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,588 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,589 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1463531938647_0001, dagName=[EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-05-18 00:38:59,590 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,595 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 74131C45C4A74C10BEBD9906E1B1F42C on vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,608 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:38:59,608 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D49754FE7ADE4CF891CDF3EC3ABE728C : {F1B99A3CCED944AB92565BA3464545E9={InputName=F1B99A3CCED944AB92565BA3464545E9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:38:59,608 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: F1B99A3CCED944AB92565BA3464545E9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:38:59,609 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,610 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,611 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:38:59,611 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:38:59,620 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: F1B99A3CCED944AB92565BA3464545E9 on vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,623 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] submitted tez dag to app master: application_1463531938647_0001, with dag id: dag_1463531938647_0001_1
2016-05-18 00:38:59,626 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 74131C45C4A74C10BEBD9906E1B1F42C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:38:59,641 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:38:59,642 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:38:59,650 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input F1B99A3CCED944AB92565BA3464545E9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:38:59,658 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:38:59,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531938647_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:38:59,664 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83], numAdditionalOutputs=1
2016-05-18 00:38:59,664 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=6DA31677E3C44DD6896B75C69A84FAEC, vertex=vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:38:59,677 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:38:59,678 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:38:59,679 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:38:59,678 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:38:59,680 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 32599 Min splitLength: 52428800 New desired splits: 1 Total length: 32599 Original splits: 2
2016-05-18 00:38:59,681 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:38:59,681 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 32599 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 32599 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:38:59,682 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:38:59,683 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F4063F9B39E84148ACD64DEDFEDA3D83:6DA31677E3C44DD6896B75C69A84FAEC using oldmapred API
2016-05-18 00:38:59,687 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=6DA31677E3C44DD6896B75C69A84FAEC, vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:38:59,696 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: F1B99A3CCED944AB92565BA3464545E9 on vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,696 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 74131C45C4A74C10BEBD9906E1B1F42C on vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, vertexId=vertex_1463531938647_0001_1_02, initRequestedTime=1463531939611, initedTime=1463531939664, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:38:59,701 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:38:59,713 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:38:59,714 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:38:59,714 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {74131C45C4A74C10BEBD9906E1B1F42C=forAllWorkUnits=true, update=[1]}
2016-05-18 00:38:59,714 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {F1B99A3CCED944AB92565BA3464545E9=forAllWorkUnits=true, update=[1]}
2016-05-18 00:38:59,717 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] parallelism set to 1
2016-05-18 00:38:59,717 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:38:59,717 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] parallelism set to 1
2016-05-18 00:38:59,717 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:38:59,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, vertexId=vertex_1463531938647_0001_1_01, initRequestedTime=1463531939608, initedTime=1463531939726, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:38:59,726 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] with distanceFromRoot: 0
2016-05-18 00:38:59,726 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:38:59,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, vertexId=vertex_1463531938647_0001_1_00, initRequestedTime=1463531939578, initedTime=1463531939728, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:38:59,728 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] with distanceFromRoot: 0
2016-05-18 00:38:59,728 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:38:59,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531938647_0001_1_01, startRequestedTime=1463531939611, startedTime=1463531939729
2016-05-18 00:38:59,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531938647_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:38:59,730 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:38:59,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531938647_0001_1_00, startRequestedTime=1463531939611, startedTime=1463531939731
2016-05-18 00:38:59,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531938647_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:38:59,732 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:38:59,733 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531938647_0001_1_01 for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] numStartedSources: 1 numSources: 2
2016-05-18 00:38:59,733 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] numStartedSources: 1 numSources: 2
2016-05-18 00:38:59,734 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531938647_0001_1_00 for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] numStartedSources: 2 numSources: 2
2016-05-18 00:38:59,734 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] with distanceFromRoot: 1
2016-05-18 00:38:59,735 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D49754FE7ADE4CF891CDF3EC3ABE728C
2016-05-18 00:38:59,736 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531938647_0001_1_02, startRequestedTime=1463531939734, startedTime=1463531939735
2016-05-18 00:38:59,738 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:38:59,737 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D3E636A48C8A4BA5B292FCD269073158
2016-05-18 00:38:59,742 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_STARTED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, taskId=task_1463531938647_0001_1_01_000000, scheduledTime=1463531939749, launchTime=1463531939749
2016-05-18 00:38:59,754 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F4063F9B39E84148ACD64DEDFEDA3D83 with 0 source tasks and 1 pending tasks
2016-05-18 00:38:59,756 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D49754FE7ADE4CF891CDF3EC3ABE728C in vertex: F4063F9B39E84148ACD64DEDFEDA3D83 numBipartiteSourceTasks: 1
2016-05-18 00:38:59,757 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D3E636A48C8A4BA5B292FCD269073158 in vertex: F4063F9B39E84148ACD64DEDFEDA3D83 numBipartiteSourceTasks: 2
2016-05-18 00:38:59,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_STARTED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, taskId=task_1463531938647_0001_1_00_000000, scheduledTime=1463531939776, launchTime=1463531939776
2016-05-18 00:38:59,779 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:38:59,779 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:38:59,819 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:38:59,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531938647_0001_00_000001, launchTime=1463531939842
2016-05-18 00:38:59,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531938647_0001_00_000002, launchTime=1463531939842
2016-05-18 00:38:59,851 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000001
2016-05-18 00:38:59,851 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000002
2016-05-18 00:38:59,858 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531938647_0001_1_00_000000_0] started. Is using containerId: [container_1463531938647_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:38:59,858 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:38:59,859 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:38:59 Starting to run new task attempt: attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:38:59 Starting to run new task attempt: attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:38:59,859 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:38:59,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:38:59,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:38:59,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0, startTime=1463531939856, containerId=container_1463531938647_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:38:59,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:38:59,869 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:38:59,869 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531938647_0001_1_01_000000_0] started. Is using containerId: [container_1463531938647_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:38:59,870 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:38:59,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0, startTime=1463531939869, containerId=container_1463531938647_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:38:59,871 INFO counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-18 00:38:59,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: D3E636A48C8A4BA5B292FCD269073158, VertexParallelism: 1, TaskAttemptID:attempt_1463531938647_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=74131C45C4A74C10BEBD9906E1B1F42C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4063F9B39E84148ACD64DEDFEDA3D83, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:38:59,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: D49754FE7ADE4CF891CDF3EC3ABE728C, VertexParallelism: 1, TaskAttemptID:attempt_1463531938647_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F1B99A3CCED944AB92565BA3464545E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4063F9B39E84148ACD64DEDFEDA3D83, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:38:59,884 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:38:59,884 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:38:59,889 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:38:59,891 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:38:59,894 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531938647_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:38:59,895 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531938647_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 00:38:59,896 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531938647_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:38:59,896 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531938647_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:38:59,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:38:59,931 INFO input.MRInput (MRInput.java:initialize(440)) - F1B99A3CCED944AB92565BA3464545E9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:38:59,934 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: F1B99A3CCED944AB92565BA3464545E9
2016-05-18 00:38:59,933 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:38:59,947 INFO input.MRInput (MRInput.java:initialize(440)) - 74131C45C4A74C10BEBD9906E1B1F42C using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:38:59,948 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 74131C45C4A74C10BEBD9906E1B1F42C
2016-05-18 00:38:59,971 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D49754FE7ADE4CF891CDF3EC3ABE728C, ordinal: 1
2016-05-18 00:38:59,973 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D49754FE7ADE4CF891CDF3EC3ABE728C, mem on start (mb), free: 187, total: 281, max: 672
2016-05-18 00:38:59,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:38:59,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:38:59,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:38:59,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:38:59,976 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F1B99A3CCED944AB92565BA3464545E9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F4063F9B39E84148ACD64DEDFEDA3D83:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:38:59,977 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D3E636A48C8A4BA5B292FCD269073158, ordinal: 0
2016-05-18 00:38:59,977 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3E636A48C8A4BA5B292FCD269073158, mem on start (mb), free: 187, total: 281, max: 672
2016-05-18 00:38:59,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:38:59,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:38:59,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:38:59,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:38:59,978 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[74131C45C4A74C10BEBD9906E1B1F42C:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F4063F9B39E84148ACD64DEDFEDA3D83:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:38:59,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:38:59,983 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:38:59,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:38:59,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:38:59,989 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[74131C45C4A74C10BEBD9906E1B1F42C:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F4063F9B39E84148ACD64DEDFEDA3D83:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:38:59,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:38:59,991 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:38:59,991 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:38:59,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 74131C45C4A74C10BEBD9906E1B1F42C
2016-05-18 00:38:59,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 74131C45C4A74C10BEBD9906E1B1F42C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:38:59,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:38:59,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:38:59,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:38:59,991 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F1B99A3CCED944AB92565BA3464545E9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F4063F9B39E84148ACD64DEDFEDA3D83:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:38:59,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:38:59,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F1B99A3CCED944AB92565BA3464545E9
2016-05-18 00:38:59,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F1B99A3CCED944AB92565BA3464545E9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:38:59,995 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:38:59,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:38:59,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:38:59,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:38:59,997 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:39:00,019 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt:0+16299, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt:16299+16300], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=32599}
2016-05-18 00:39:00,019 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:39:00,023 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 74131C45C4A74C10BEBD9906E1B1F42C initialized RecordReader from event
2016-05-18 00:39:00,022 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - F1B99A3CCED944AB92565BA3464545E9 initialized RecordReader from event
2016-05-18 00:39:00,045 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:39:00,045 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:39:00,082 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"] streamed: true, id: F1B99A3CCED944AB92565BA3464545E9
2016-05-18 00:39:00,082 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 74131C45C4A74C10BEBD9906E1B1F42C
2016-05-18 00:39:00,083 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 022443DF8A1543E49F86CC35ACE7D3E8
2016-05-18 00:39:00,083 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 022443DF8A1543E49F86CC35ACE7D3E8
2016-05-18 00:39:00,083 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 022443DF8A1543E49F86CC35ACE7D3E8
2016-05-18 00:39:00,085 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 022443DF8A1543E49F86CC35ACE7D3E8
2016-05-18 00:39:00,135 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F4063F9B39E84148ACD64DEDFEDA3D83 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4caabc8c, comparator=cascading.tuple.hadoop.util.TupleComparator@4290e01e, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:00,135 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F4063F9B39E84148ACD64DEDFEDA3D83 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@5d40b2a6, comparator=cascading.tuple.hadoop.util.TupleComparator@3569f3b6, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:00,142 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F4063F9B39E84148ACD64DEDFEDA3D83: , UsingHashComparator=false
2016-05-18 00:39:00,142 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F4063F9B39E84148ACD64DEDFEDA3D83: , UsingHashComparator=false
2016-05-18 00:39:00,250 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:00,252 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:00,253 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F4063F9B39E84148ACD64DEDFEDA3D83: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:00,255 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F4063F9B39E84148ACD64DEDFEDA3D83: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:00,261 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:39:00,263 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D3E636A48C8A4BA5B292FCD269073158, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:00,356 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:00,356 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:00,357 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F4063F9B39E84148ACD64DEDFEDA3D83: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:00,357 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F4063F9B39E84148ACD64DEDFEDA3D83: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:00,358 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:39:00,362 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D49754FE7ADE4CF891CDF3EC3ABE728C, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:00,366 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:39:00,366 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D3E636A48C8A4BA5B292FCD269073158, mem on close (mb), free: 75, total: 382, max: 672
2016-05-18 00:39:00,368 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F4063F9B39E84148ACD64DEDFEDA3D83: Starting flush of map output
2016-05-18 00:39:00,368 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F4063F9B39E84148ACD64DEDFEDA3D83: Span0.length = 20, perItem = 5
2016-05-18 00:39:00,372 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F4063F9B39E84148ACD64DEDFEDA3D83: done sorting span=0, length=20, time=4
2016-05-18 00:39:00,375 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F4063F9B39E84148ACD64DEDFEDA3D83: Heap = SpanIterator<0:19> (span=Span[16777216,102]),
2016-05-18 00:39:00,380 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F4063F9B39E84148ACD64DEDFEDA3D83: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-local-dir/output/attempt_1463531938647_0001_1_00_000000_0_10005_0/file.out
2016-05-18 00:39:00,417 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:39:00,417 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D49754FE7ADE4CF891CDF3EC3ABE728C, mem on close (mb), free: 72, total: 382, max: 672
2016-05-18 00:39:00,418 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F4063F9B39E84148ACD64DEDFEDA3D83: Starting flush of map output
2016-05-18 00:39:00,418 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F4063F9B39E84148ACD64DEDFEDA3D83: Span0.length = 200, perItem = 45
2016-05-18 00:39:00,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:00 Completed running task attempt: attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:39:00,424 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:00,425 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531938647_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:00,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531938647_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531940263, Process_Duration=103, Process_End_Time=1463531940366, Read_Duration=93, Tuples_Read=20, Tuples_Written=20, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 00:39:00,437 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531938647_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,438 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, taskAttemptId=attempt_1463531938647_0001_1_00_000000_0, creationTime=1463531939776, allocationTime=1463531939830, startTime=1463531939856, finishTime=1463531940436, timeTaken=580, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:00,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:00,440 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:00,441 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000002
2016-05-18 00:39:00,444 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F4063F9B39E84148ACD64DEDFEDA3D83: done sorting span=0, length=200, time=26
2016-05-18 00:39:00,444 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F4063F9B39E84148ACD64DEDFEDA3D83: Heap = SpanIterator<0:199> (span=Span[16777216,9061]),
2016-05-18 00:39:00,450 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F4063F9B39E84148ACD64DEDFEDA3D83: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-local-dir/output/attempt_1463531938647_0001_1_01_000000_0_10006_0/file.out
2016-05-18 00:39:00,455 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531938647_0001_1_00_000000_0
2016-05-18 00:39:00,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_FINISHED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, taskId=task_1463531938647_0001_1_00_000000, startTime=1463531939856, finishTime=1463531940457, timeTaken=601, status=SUCCEEDED, successfulAttemptID=attempt_1463531938647_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531940263, Process_Duration=103, Process_End_Time=1463531940366, Read_Duration=93, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 00:39:00,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531938647_0001_00_000002, stoppedTime=1463531940469, exitStatus=0
2016-05-18 00:39:00,474 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531938647_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:00,476 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000002: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:00,476 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531938647_0001_00_000002, Exiting
2016-05-18 00:39:00,476 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531938647_0001_00_000002
2016-05-18 00:39:00,476 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531938647_0001_00_000002
2016-05-18 00:39:00,475 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:00,477 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:39:00,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:00 Completed running task attempt: attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:39:00,478 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:00,479 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531938647_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:00,479 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531938647_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:00,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531938647_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531940362, Process_Duration=55, Process_End_Time=1463531940417, Read_Duration=7, Tuples_Read=200, Tuples_Written=200, Write_Duration=21][cascading.flow.StepCounters Tuples_Read=200]]
2016-05-18 00:39:00,480 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531938647_0001_00_000002 completed successfully
2016-05-18 00:39:00,485 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531938647_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,486 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,486 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:00,486 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:00,487 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000001
2016-05-18 00:39:00,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_FINISHED]: vertexName=D3E636A48C8A4BA5B292FCD269073158, vertexId=vertex_1463531938647_0001_1_00, initRequestedTime=1463531939578, initedTime=1463531939728, startRequestedTime=1463531939611, startedTime=1463531939731, finishTime=1463531940477, timeTaken=746, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531940263, Process_Duration=103, Process_End_Time=1463531940366, Read_Duration=93, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463531939856, firstTasksToStart=[ task_1463531938647_0001_1_00_000000 ], lastTaskFinishTime=1463531940436, lastTasksToFinish=[ task_1463531938647_0001_1_00_000000 ], minTaskDuration=580, maxTaskDuration=580, avgTaskDuration=580.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531938647_0001_1_00_000000 ], longestDurationTasks=[ task_1463531938647_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:00,500 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:00,501 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463531938647_0001_00_000002
2016-05-18 00:39:00,501 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] attempt: attempt_1463531938647_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:00,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, taskAttemptId=attempt_1463531938647_0001_1_01_000000_0, creationTime=1463531939771, allocationTime=1463531939829, startTime=1463531939869, finishTime=1463531940503, timeTaken=634, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:00,504 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:00,504 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:00,505 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531938647_0001_1_01_000000_0
2016-05-18 00:39:00,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_FINISHED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, taskId=task_1463531938647_0001_1_01_000000, startTime=1463531939869, finishTime=1463531940505, timeTaken=636, status=SUCCEEDED, successfulAttemptID=attempt_1463531938647_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531940362, Process_Duration=55, Process_End_Time=1463531940417, Read_Duration=7, Tuples_Read=200, Tuples_Written=200, Write_Duration=21, cascading.flow.StepCounters, Tuples_Read=200
2016-05-18 00:39:00,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531938647_0001_00_000001, stoppedTime=1463531940507, exitStatus=0
2016-05-18 00:39:00,508 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:00,508 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:39:00,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_FINISHED]: vertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, vertexId=vertex_1463531938647_0001_1_01, initRequestedTime=1463531939608, initedTime=1463531939726, startRequestedTime=1463531939611, startedTime=1463531939729, finishTime=1463531940508, timeTaken=779, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531940362, Process_Duration=55, Process_End_Time=1463531940417, Read_Duration=7, Tuples_Read=200, Tuples_Written=200, Write_Duration=21, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1463531939869, firstTasksToStart=[ task_1463531938647_0001_1_01_000000 ], lastTaskFinishTime=1463531940503, lastTasksToFinish=[ task_1463531938647_0001_1_01_000000 ], minTaskDuration=634, maxTaskDuration=634, avgTaskDuration=634.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531938647_0001_1_01_000000 ], longestDurationTasks=[ task_1463531938647_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:00,510 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:00,510 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531938647_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:00,510 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] attempt: attempt_1463531938647_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:00,513 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F4063F9B39E84148ACD64DEDFEDA3D83
2016-05-18 00:39:00,514 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, destinationVertexName=F4063F9B39E84148ACD64DEDFEDA3D83. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:00,514 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D3E636A48C8A4BA5B292FCD269073158, destinationVertexName=F4063F9B39E84148ACD64DEDFEDA3D83. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:00,514 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:39:00,515 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:00,515 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:00,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_STARTED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, taskId=task_1463531938647_0001_1_02_000000, scheduledTime=1463531940515, launchTime=1463531940515
2016-05-18 00:39:00,516 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:39:00,518 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531938647_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:00,520 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000001: 33 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:00,520 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531938647_0001_00_000001, Exiting
2016-05-18 00:39:00,520 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531938647_0001_00_000001
2016-05-18 00:39:00,520 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531938647_0001_00_000001
2016-05-18 00:39:00,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531938647_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:00,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531938647_0001_00_000001 completed successfully
2016-05-18 00:39:00,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531938647_0001_00_000003, launchTime=1463531940521
2016-05-18 00:39:00,522 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000003
2016-05-18 00:39:00,523 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531938647_0001_1_02_000000_0] started. Is using containerId: [container_1463531938647_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 00:39:00,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0, startTime=1463531940523, containerId=container_1463531938647_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 00:39:00,524 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00 Starting to run new task attempt: attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,525 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,526 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:00,526 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:00,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: F4063F9B39E84148ACD64DEDFEDA3D83, VertexParallelism: 1, TaskAttemptID:attempt_1463531938647_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D49754FE7ADE4CF891CDF3EC3ABE728C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=D3E636A48C8A4BA5B292FCD269073158, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6DA31677E3C44DD6896B75C69A84FAEC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:00,526 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:00,526 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,529 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531938647_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:39:00,529 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531938647_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 00:39:00,535 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:00,548 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:00,549 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:00,560 INFO output.MROutput (MROutput.java:initialize(421)) - 6DA31677E3C44DD6896B75C69A84FAEC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:00,564 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4063F9B39E84148ACD64DEDFEDA3D83, ordinal: 2
2016-05-18 00:39:00,564 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4063F9B39E84148ACD64DEDFEDA3D83, mem on start (mb), free: 59, total: 382, max: 672
2016-05-18 00:39:00,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:00,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 00:39:00,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:00,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:00,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:00,565 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[6DA31677E3C44DD6896B75C69A84FAEC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [D3E636A48C8A4BA5B292FCD269073158:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [D49754FE7ADE4CF891CDF3EC3ABE728C:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:00,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:00,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 00:39:00,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 00:39:00,567 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[6DA31677E3C44DD6896B75C69A84FAEC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [D3E636A48C8A4BA5B292FCD269073158:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [D49754FE7ADE4CF891CDF3EC3ABE728C:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:00,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:00,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D49754FE7ADE4CF891CDF3EC3ABE728C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:00,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D3E636A48C8A4BA5B292FCD269073158 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:00,569 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D49754FE7ADE4CF891CDF3EC3ABE728C: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:00,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 00:39:00,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 00:39:00,572 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D3E636A48C8A4BA5B292FCD269073158: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:00,573 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D3E636A48C8A4BA5B292FCD269073158: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:00,573 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D49754FE7ADE4CF891CDF3EC3ABE728C: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:00,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D3E636A48C8A4BA5B292FCD269073158 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:00,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D49754FE7ADE4CF891CDF3EC3ABE728C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:00,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D49754FE7ADE4CF891CDF3EC3ABE728C
2016-05-18 00:39:00,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D3E636A48C8A4BA5B292FCD269073158
2016-05-18 00:39:00,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:00,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:00,597 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,605 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-05-18 00:39:00,607 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D3E636A48C8A4BA5B292FCD269073158: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:00,607 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D49754FE7ADE4CF891CDF3EC3ABE728C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:00,611 WARN util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@6c7bbfee, and: cascading.platform.hadoop.TestLongComparator@3075d9e6
2016-05-18 00:39:00,611 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: 022443DF8A1543E49F86CC35ACE7D3E8
2016-05-18 00:39:00,611 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"], id: 6DA31677E3C44DD6896B75C69A84FAEC
2016-05-18 00:39:00,612 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-05-18 00:39:00,612 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531938647_0001_1_00_000000_0_10005} to DISK_DIRECT, csize=142, dsize=138, EndTime=1463531940612, TimeTaken=4, Rate=0.03 MB/s
2016-05-18 00:39:00,613 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D3E636A48C8A4BA5B292FCD269073158
2016-05-18 00:39:00,613 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 022443DF8A1543E49F86CC35ACE7D3E8, for 2 inputs
2016-05-18 00:39:00,613 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 022443DF8A1543E49F86CC35ACE7D3E8, for 2 inputs
2016-05-18 00:39:00,613 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:00,614 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D3E636A48C8A4BA5B292FCD269073158, wasInterrupted=false
2016-05-18 00:39:00,614 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531938647_0001_1_01_000000_0_10006} to DISK_DIRECT, csize=8931, dsize=8927, EndTime=1463531940614, TimeTaken=4, Rate=2.13 MB/s
2016-05-18 00:39:00,615 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D49754FE7ADE4CF891CDF3EC3ABE728C
2016-05-18 00:39:00,615 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:00,615 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-05-18 00:39:00,615 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D49754FE7ADE4CF891CDF3EC3ABE728C, wasInterrupted=false
2016-05-18 00:39:00,616 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:00,616 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-05-18 00:39:00,617 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - D49754FE7ADE4CF891CDF3EC3ABE728C: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:00,617 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:00,618 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-local-dir/output/attempt_1463531938647_0001_1_00_000000_0_10005/file.out, len=142, isLocal=true
2016-05-18 00:39:00,618 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001_wd/localmode-local-dir/output/attempt_1463531938647_0001_1_01_000000_0_10006/file.out, len=8931, isLocal=true
2016-05-18 00:39:00,619 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 8931 bytes from disk
2016-05-18 00:39:00,619 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 142 bytes from disk
2016-05-18 00:39:00,620 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:00,620 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:00,623 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 00:39:00,623 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 00:39:00,625 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:00,625 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:00,672 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 8917 bytes
2016-05-18 00:39:00,672 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 128 bytes
2016-05-18 00:39:00,673 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D49754FE7ADE4CF891CDF3EC3ABE728C
2016-05-18 00:39:00,674 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4063F9B39E84148ACD64DEDFEDA3D83, all 2 inputs ready in: 00:00:00.011
2016-05-18 00:39:00,673 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D3E636A48C8A4BA5B292FCD269073158
2016-05-18 00:39:00,676 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D3E636A48C8A4BA5B292FCD269073158: Shuffle Runner thread complete
2016-05-18 00:39:00,676 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D49754FE7ADE4CF891CDF3EC3ABE728C: Shuffle Runner thread complete
2016-05-18 00:39:00,677 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D49754FE7ADE4CF891CDF3EC3ABE728C: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:00,687 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D3E636A48C8A4BA5B292FCD269073158: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:00,726 INFO output.MROutput (MROutput.java:close(526)) - 6DA31677E3C44DD6896B75C69A84FAEC closed
2016-05-18 00:39:00,728 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531938647_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 00:39:00,728 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,729 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4063F9B39E84148ACD64DEDFEDA3D83, mem on close (mb), free: 267, total: 498, max: 672
2016-05-18 00:39:00,729 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D49754FE7ADE4CF891CDF3EC3ABE728C
2016-05-18 00:39:00,729 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D49754FE7ADE4CF891CDF3EC3ABE728C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:00,729 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D3E636A48C8A4BA5B292FCD269073158
2016-05-18 00:39:00,729 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D3E636A48C8A4BA5B292FCD269073158: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:00,730 INFO output.MROutput (MROutput.java:close(526)) - 6DA31677E3C44DD6896B75C69A84FAEC closed
2016-05-18 00:39:00,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:00 Completed running task attempt: attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,730 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0, askedToStop=false
2016-05-18 00:39:00,730 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531938647_0001_1_02_000000_0, stopRequested=false
2016-05-18 00:39:00,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531938647_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=89, MERGE_PHASE_TIME=204, FIRST_EVENT_RECEIVED=73, LAST_EVENT_RECEIVED=73][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531940674, Process_Duration=54, Process_End_Time=1463531940728, Read_Duration=3, Tuples_Read=124, Tuples_Written=199, Write_Duration=3][cascading.flow.StepCounters Tuples_Written=199]]
2016-05-18 00:39:00,733 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531938647_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,733 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:00,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, taskAttemptId=attempt_1463531938647_0001_1_02_000000_0, creationTime=1463531940516, allocationTime=1463531940521, startTime=1463531940523, finishTime=1463531940733, timeTaken=210, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:00,733 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:00,734 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531938647_0001_1_02_000000_0
2016-05-18 00:39:00,736 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:00,737 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531938647_0001_00_000003
2016-05-18 00:39:00,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:TASK_FINISHED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, taskId=task_1463531938647_0001_1_02_000000, startTime=1463531940523, finishTime=1463531940736, timeTaken=213, status=SUCCEEDED, successfulAttemptID=attempt_1463531938647_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=89, MERGE_PHASE_TIME=204, FIRST_EVENT_RECEIVED=73, LAST_EVENT_RECEIVED=73, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531940674, Process_Duration=54, Process_End_Time=1463531940728, Read_Duration=3, Tuples_Read=124, Tuples_Written=199, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=199
2016-05-18 00:39:00,740 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:00,740 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83]
2016-05-18 00:39:00,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:VERTEX_FINISHED]: vertexName=F4063F9B39E84148ACD64DEDFEDA3D83, vertexId=vertex_1463531938647_0001_1_02, initRequestedTime=1463531939611, initedTime=1463531939664, startRequestedTime=1463531939734, startedTime=1463531939735, finishTime=1463531940740, timeTaken=1005, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=89, MERGE_PHASE_TIME=204, FIRST_EVENT_RECEIVED=73, LAST_EVENT_RECEIVED=73, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531940674, Process_Duration=54, Process_End_Time=1463531940728, Read_Duration=3, Tuples_Read=124, Tuples_Written=199, Write_Duration=3, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1463531940523, firstTasksToStart=[ task_1463531938647_0001_1_02_000000 ], lastTaskFinishTime=1463531940733, lastTasksToFinish=[ task_1463531938647_0001_1_02_000000 ], minTaskDuration=210, maxTaskDuration=210, avgTaskDuration=210.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531938647_0001_1_02_000000 ], longestDurationTasks=[ task_1463531938647_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:00,744 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:00,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531938647_0001_00_000003, stoppedTime=1463531940745, exitStatus=0
2016-05-18 00:39:00,745 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:00,745 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:00,746 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531938647_0001_1_00 [D3E636A48C8A4BA5B292FCD269073158]
2016-05-18 00:39:00,746 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531938647_0001_1_01 [D49754FE7ADE4CF891CDF3EC3ABE728C]
2016-05-18 00:39:00,748 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531938647_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 00:39:00,748 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531938647_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:00,749 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531938647_0001_00_000003, Exiting
2016-05-18 00:39:00,749 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531938647_0001_00_000003
2016-05-18 00:39:00,749 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531938647_0001_00_000003
2016-05-18 00:39:00,749 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531938647_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:00,749 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531938647_0001_00_000003 completed successfully
2016-05-18 00:39:00,750 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531938647_0001_1
2016-05-18 00:39:00,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531938647_0001_1
2016-05-18 00:39:00,758 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 6DA31677E3C44DD6896B75C69A84FAEC for vertex: vertex_1463531938647_0001_1_02 [F4063F9B39E84148ACD64DEDFEDA3D83], outputName: 6DA31677E3C44DD6896B75C69A84FAEC
2016-05-18 00:39:00,758 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531938647_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:00,762 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463531938647_0001_00_000003
2016-05-18 00:39:00,771 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:6DA31677E3C44DD6896B75C69A84FAEC of vertex/vertexGroup:F4063F9B39E84148ACD64DEDFEDA3D83 isVertexGroupOutput:false
2016-05-18 00:39:00,771 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:00,781 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531938647_0001_1, queueSize=0
2016-05-18 00:39:00,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531938647_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531938647_0001_1, startTime=1463531939575, finishTime=1463531940771, timeTaken=1196, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=2860, AM_GC_TIME_MILLIS=35, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=89, MERGE_PHASE_TIME=204, FIRST_EVENT_RECEIVED=73, LAST_EVENT_RECEIVED=73, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390595821299, Process_Duration=212, Process_End_Time=4390595821511, Read_Duration=103, Tuples_Read=344, Tuples_Written=419, Write_Duration=27, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-05-18 00:39:00,785 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531938647_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:00,785 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531938647_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:00 Completed Dag: dag_1463531938647_0001_1
2016-05-18 00:39:00,786 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531938647_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:00,787 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531938647_0001_1_post
2016-05-18 00:39:00,787 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:00,788 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:00,788 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:00,790 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed, with id=dag_1463531938647_0001_1
2016-05-18 00:39:00,796 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B] apache*ip/(1/1) ...ed/cogroupcomparereversed, with id=dag_1463531938647_0001_1
2016-05-18 00:39:00,797 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1463531938647_0001
2016-05-18 00:39:00,802 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:00,803 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:00,803 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:00,803 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:00,803 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:00,804 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:00,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:00,807 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:00,808 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:00,808 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:00,809 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:00,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:00,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:00,810 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:00,814 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EB7661AC92D84352A8874BE7893C5936/F004D338D6C34325ACE876579A09345B/.tez/application_1463531938647_0001
2016-05-18 00:39:00,815 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed/_temporary
2016-05-18 00:39:00,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] completed in: 00:02.376
2016-05-18 00:39:00,903 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-05-18 00:39:00,960 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:00,961 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:01,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-18 00:39:01,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.086
2016-05-18 00:39:01,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 00:39:01,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-05-18 00:39:01,089 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:01,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt
2016-05-18 00:39:01,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt
2016-05-18 00:39:01,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-05-18 00:39:01,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] starting
2016-05-18 00:39:01,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:39:01,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:39:01,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-05-18 00:39:01,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] parallel execution of steps is enabled: false
2016-05-18 00:39:01,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] executing total steps: 1
2016-05-18 00:39:01,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] allocating management threads: 1
2016-05-18 00:39:01,283 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] starting step: (1/1) ...ues/cogroupcompareforward
2016-05-18 00:39:01,283 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:01,305 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] tez session mode enabled: true
2016-05-18 00:39:01,311 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:01,312 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:01,312 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:01,323 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001 doesn't exist and is created
2016-05-18 00:39:01,342 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:01,342 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:01,344 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd
2016-05-18 00:39:01,356 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531941312_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:01,356 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:01,357 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:01,358 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:01,358 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:01,358 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:01,358 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:01,359 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:01,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:01,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:01,362 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:01,363 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:01,363 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:01,363 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:01,363 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:01,364 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:01,364 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:01,364 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:01,364 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:01,364 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:01,367 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:01,369 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:01,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:01,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-log-dir
2016-05-18 00:39:01,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-log-dir/history.txt.appattempt_1463531941312_0001_000000, maxErrors=10
2016-05-18 00:39:01,370 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:01,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531941312_0001_000000, appSubmitTime=1463531941355, launchTime=1463531941356
2016-05-18 00:39:01,378 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38790
2016-05-18 00:39:01,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:01,382 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531941312
2016-05-18 00:39:01,389 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531941312_0001_000000, startTime=1463531941388
2016-05-18 00:39:01,389 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:01,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:01,443 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:01,443 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1463531941312_0001, dagName=[ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-05-18 00:39:01,466 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-05-18 00:39:01,466 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:01,467 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531941312_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-log-dir/dag_1463531941312_0001_1.dot
2016-05-18 00:39:01,471 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-log-dir/dag_1463531941312_0001_1-tez-dag.pb.txt
2016-05-18 00:39:01,485 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531941312_0001_1
2016-05-18 00:39:01,485 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,485 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-05-18 00:39:01 Running Dag: dag_1463531941312_0001_1
2016-05-18 00:39:01,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531941312_0001_1, submitTime=1463531941466
2016-05-18 00:39:01,499 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001/tez-conf.pb" } size: 36903 timestamp: 1463531941000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:01,499 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,499 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:01,500 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001/tez-conf.pb" } size: 36903 timestamp: 1463531941000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:01,500 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,501 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:01,501 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001/tez-conf.pb" } size: 36903 timestamp: 1463531941000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:01,501 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,502 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:01,502 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:01,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531941312_0001_1, initTime=1463531941502
2016-05-18 00:39:01,503 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531941312_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:01,504 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:01,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:DAG_STARTED]: dagID=dag_1463531941312_0001_1, startTime=1463531941504
2016-05-18 00:39:01,504 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531941312_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:01,504 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: EE038F2E8E584D58B0DEBBDDC84D453D : {76970CC5C05D40239CAC9A70F716FD20={InputName=76970CC5C05D40239CAC9A70F716FD20}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:01,504 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 76970CC5C05D40239CAC9A70F716FD20, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:01,505 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,505 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,506 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:01,506 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 76970CC5C05D40239CAC9A70F716FD20 on vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,506 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: EEE5AEA24D5D42C89B76A913A579A1B3 : {FAF46A1800AC46B2BB6D7750B55AB1E3={InputName=FAF46A1800AC46B2BB6D7750B55AB1E3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:01,506 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FAF46A1800AC46B2BB6D7750B55AB1E3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:01,506 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,506 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,507 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,507 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,507 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:01,508 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1463531941312_0001, dagName=[ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-05-18 00:39:01,508 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,508 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FAF46A1800AC46B2BB6D7750B55AB1E3 on vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,510 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 76970CC5C05D40239CAC9A70F716FD20 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:01,515 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FAF46A1800AC46B2BB6D7750B55AB1E3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:01,516 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:01,518 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:01,518 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,518 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 275 Min splitLength: 52428800 New desired splits: 1 Total length: 275 Original splits: 2
2016-05-18 00:39:01,519 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 275 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 275 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:01,519 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:01,521 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 76970CC5C05D40239CAC9A70F716FD20 on vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,521 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531941312_0001_1_02, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:01,521 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71], numAdditionalOutputs=1
2016-05-18 00:39:01,522 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=89D7CE821C8640B0B2D2430CCBEDEF1E, vertex=vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:01,526 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F470605B72BF427BB63906D14BCEFB71:89D7CE821C8640B0B2D2430CCBEDEF1E using oldmapred API
2016-05-18 00:39:01,531 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=89D7CE821C8640B0B2D2430CCBEDEF1E, vertexName=F470605B72BF427BB63906D14BCEFB71, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:01,530 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:01,536 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 32599 Min splitLength: 52428800 New desired splits: 1 Total length: 32599 Original splits: 2
2016-05-18 00:39:01,536 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 32599 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 32599 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:01,536 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:01,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F470605B72BF427BB63906D14BCEFB71, vertexId=vertex_1463531941312_0001_1_02, initRequestedTime=1463531941508, initedTime=1463531941521, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:01,538 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:01,540 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FAF46A1800AC46B2BB6D7750B55AB1E3 on vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,541 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:01,542 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:01,542 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {76970CC5C05D40239CAC9A70F716FD20=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:01,543 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FAF46A1800AC46B2BB6D7750B55AB1E3=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:01,543 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] parallelism set to 1
2016-05-18 00:39:01,545 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:01,544 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] parallelism set to 1
2016-05-18 00:39:01,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, vertexId=vertex_1463531941312_0001_1_01, initRequestedTime=1463531941504, initedTime=1463531941545, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:01,546 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] with distanceFromRoot: 0
2016-05-18 00:39:01,547 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:01,546 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:01,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531941312_0001_1_01, startRequestedTime=1463531941508, startedTime=1463531941547
2016-05-18 00:39:01,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531941312_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:01,549 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:01,549 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in EE038F2E8E584D58B0DEBBDDC84D453D
2016-05-18 00:39:01,549 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, vertexId=vertex_1463531941312_0001_1_00, initRequestedTime=1463531941506, initedTime=1463531941549, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:01,550 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] with distanceFromRoot: 0
2016-05-18 00:39:01,551 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:01,551 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531941312_0001_1_01 for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] numStartedSources: 1 numSources: 2
2016-05-18 00:39:01,551 INFO impl.VertexImpl (VertexImpl.java:transition(2937)) - Cannot start vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] numStartedSources: 1 numSources: 2
2016-05-18 00:39:01,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_STARTED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, taskId=task_1463531941312_0001_1_01_000000, scheduledTime=1463531941552, launchTime=1463531941552
2016-05-18 00:39:01,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531941312_0001_1_00, startRequestedTime=1463531941508, startedTime=1463531941554
2016-05-18 00:39:01,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531941312_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:01,556 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:01,556 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in EEE5AEA24D5D42C89B76A913A579A1B3
2016-05-18 00:39:01,556 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,556 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,557 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531941312_0001_1_00 for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] numStartedSources: 2 numSources: 2
2016-05-18 00:39:01,557 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] with distanceFromRoot: 1
2016-05-18 00:39:01,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531941312_0001_1_02, startRequestedTime=1463531941557, startedTime=1463531941560
2016-05-18 00:39:01,561 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:01,562 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F470605B72BF427BB63906D14BCEFB71 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:01,563 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: EE038F2E8E584D58B0DEBBDDC84D453D in vertex: F470605B72BF427BB63906D14BCEFB71 numBipartiteSourceTasks: 1
2016-05-18 00:39:01,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_STARTED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, taskId=task_1463531941312_0001_1_00_000000, scheduledTime=1463531941561, launchTime=1463531941561
2016-05-18 00:39:01,563 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: EEE5AEA24D5D42C89B76A913A579A1B3 in vertex: F470605B72BF427BB63906D14BCEFB71 numBipartiteSourceTasks: 2
2016-05-18 00:39:01,565 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] submitted tez dag to app master: application_1463531941312_0001, with dag id: dag_1463531941312_0001_1
2016-05-18 00:39:01,569 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:01,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531941312_0001_00_000001, launchTime=1463531941574
2016-05-18 00:39:01,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531941312_0001_00_000002, launchTime=1463531941575
2016-05-18 00:39:01,576 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000001
2016-05-18 00:39:01,578 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531941312_0001_1_01_000000_0] started. Is using containerId: [container_1463531941312_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:01,578 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000002
2016-05-18 00:39:01,579 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0, startTime=1463531941578, containerId=container_1463531941312_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:01,582 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,582 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531941312_0001_1_00_000000_0] started. Is using containerId: [container_1463531941312_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:01 Starting to run new task attempt: attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,583 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,583 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,583 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:01,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: EE038F2E8E584D58B0DEBBDDC84D453D, VertexParallelism: 1, TaskAttemptID:attempt_1463531941312_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=76970CC5C05D40239CAC9A70F716FD20, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F470605B72BF427BB63906D14BCEFB71, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:01,584 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:01,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0, startTime=1463531941582, containerId=container_1463531941312_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:01,587 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01 Starting to run new task attempt: attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,589 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,589 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531941312_0001_1_01_000000_0 sent events: (0-1).
2016-05-18 00:39:01,589 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531941312_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:01,589 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,591 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:01,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: EEE5AEA24D5D42C89B76A913A579A1B3, VertexParallelism: 1, TaskAttemptID:attempt_1463531941312_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FAF46A1800AC46B2BB6D7750B55AB1E3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F470605B72BF427BB63906D14BCEFB71, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:01,591 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:01,597 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,599 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:01,607 INFO input.MRInput (MRInput.java:initialize(440)) - 76970CC5C05D40239CAC9A70F716FD20 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:01,607 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 76970CC5C05D40239CAC9A70F716FD20
2016-05-18 00:39:01,614 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531941312_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:01,614 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531941312_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:01,616 INFO input.MRInput (MRInput.java:initialize(440)) - FAF46A1800AC46B2BB6D7750B55AB1E3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:01,616 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FAF46A1800AC46B2BB6D7750B55AB1E3
2016-05-18 00:39:01,620 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE038F2E8E584D58B0DEBBDDC84D453D, ordinal: 0
2016-05-18 00:39:01,620 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE038F2E8E584D58B0DEBBDDC84D453D, mem on start (mb), free: 119, total: 498, max: 672
2016-05-18 00:39:01,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:01,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:01,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:01,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:01,621 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[76970CC5C05D40239CAC9A70F716FD20:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F470605B72BF427BB63906D14BCEFB71:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:01,622 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:01,622 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:01,623 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:01,623 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[76970CC5C05D40239CAC9A70F716FD20:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F470605B72BF427BB63906D14BCEFB71:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:01,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:01,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 76970CC5C05D40239CAC9A70F716FD20
2016-05-18 00:39:01,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 76970CC5C05D40239CAC9A70F716FD20 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:01,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:01,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:01,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:01,626 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,632 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:01,634 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:0+137, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt:137+138], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=275}
2016-05-18 00:39:01,634 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 76970CC5C05D40239CAC9A70F716FD20 initialized RecordReader from event
2016-05-18 00:39:01,636 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:39:01,638 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 76970CC5C05D40239CAC9A70F716FD20
2016-05-18 00:39:01,638 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: DA28085F4246455196F9F400F6B7631A
2016-05-18 00:39:01,638 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] DA28085F4246455196F9F400F6B7631A
2016-05-18 00:39:01,648 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EEE5AEA24D5D42C89B76A913A579A1B3, ordinal: 1
2016-05-18 00:39:01,648 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEE5AEA24D5D42C89B76A913A579A1B3, mem on start (mb), free: 116, total: 498, max: 672
2016-05-18 00:39:01,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:01,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:01,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:01,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:01,648 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FAF46A1800AC46B2BB6D7750B55AB1E3:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F470605B72BF427BB63906D14BCEFB71:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:01,649 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:01,649 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:01,650 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:01,650 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F470605B72BF427BB63906D14BCEFB71 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@141ffc85, comparator=cascading.tuple.hadoop.util.TupleComparator@3ffb8a15, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:01,650 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FAF46A1800AC46B2BB6D7750B55AB1E3:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F470605B72BF427BB63906D14BCEFB71:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:01,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:01,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FAF46A1800AC46B2BB6D7750B55AB1E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:01,651 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F470605B72BF427BB63906D14BCEFB71: , UsingHashComparator=false
2016-05-18 00:39:01,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FAF46A1800AC46B2BB6D7750B55AB1E3
2016-05-18 00:39:01,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:01,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:01,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:01,677 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,687 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:01,688 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:01,688 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F470605B72BF427BB63906D14BCEFB71: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:01,688 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F470605B72BF427BB63906D14BCEFB71: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:01,691 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/ips.20.txt"]
2016-05-18 00:39:01,692 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE038F2E8E584D58B0DEBBDDC84D453D, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:01,695 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:39:01,697 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"] streamed: true, id: FAF46A1800AC46B2BB6D7750B55AB1E3
2016-05-18 00:39:01,698 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: DA28085F4246455196F9F400F6B7631A
2016-05-18 00:39:01,698 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] DA28085F4246455196F9F400F6B7631A
2016-05-18 00:39:01,698 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt:0+16299, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt:16299+16300], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=32599}
2016-05-18 00:39:01,700 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FAF46A1800AC46B2BB6D7750B55AB1E3 initialized RecordReader from event
2016-05-18 00:39:01,701 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,702 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE038F2E8E584D58B0DEBBDDC84D453D, mem on close (mb), free: 164, total: 498, max: 672
2016-05-18 00:39:01,704 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F470605B72BF427BB63906D14BCEFB71: Starting flush of map output
2016-05-18 00:39:01,706 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F470605B72BF427BB63906D14BCEFB71: Span0.length = 20, perItem = 5
2016-05-18 00:39:01,708 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F470605B72BF427BB63906D14BCEFB71: done sorting span=0, length=20, time=2
2016-05-18 00:39:01,709 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F470605B72BF427BB63906D14BCEFB71: Heap = SpanIterator<0:19> (span=Span[16777216,102]),
2016-05-18 00:39:01,711 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F470605B72BF427BB63906D14BCEFB71: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-local-dir/output/attempt_1463531941312_0001_1_01_000000_0_10012_0/file.out
2016-05-18 00:39:01,713 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F470605B72BF427BB63906D14BCEFB71 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@10a9420a, comparator=cascading.tuple.hadoop.util.TupleComparator@582ea711, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:01,714 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F470605B72BF427BB63906D14BCEFB71: , UsingHashComparator=false
2016-05-18 00:39:01,774 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:01,775 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:01,775 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F470605B72BF427BB63906D14BCEFB71: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:01,776 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F470605B72BF427BB63906D14BCEFB71: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:01,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:01,778 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/apache.200.txt"]
2016-05-18 00:39:01,780 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EEE5AEA24D5D42C89B76A913A579A1B3, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:01 Completed running task attempt: attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,780 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:01,782 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531941312_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:01,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531941312_0001_1_01_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531941692, Process_Duration=9, Process_End_Time=1463531941701, Read_Duration=6, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-18 00:39:01,785 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531941312_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,785 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,786 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:01,786 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,786 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000001
2016-05-18 00:39:01,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, taskAttemptId=attempt_1463531941312_0001_1_01_000000_0, creationTime=1463531941553, allocationTime=1463531941572, startTime=1463531941578, finishTime=1463531941786, timeTaken=208, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:01,789 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531941312_0001_1_01_000000_0
2016-05-18 00:39:01,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_FINISHED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, taskId=task_1463531941312_0001_1_01_000000, startTime=1463531941578, finishTime=1463531941789, timeTaken=211, status=SUCCEEDED, successfulAttemptID=attempt_1463531941312_0001_1_01_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531941692, Process_Duration=9, Process_End_Time=1463531941701, Read_Duration=6, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-18 00:39:01,791 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:01,791 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_FINISHED]: vertexName=EE038F2E8E584D58B0DEBBDDC84D453D, vertexId=vertex_1463531941312_0001_1_01, initRequestedTime=1463531941504, initedTime=1463531941545, startRequestedTime=1463531941508, startedTime=1463531941547, finishTime=1463531941791, timeTaken=244, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531941692, Process_Duration=9, Process_End_Time=1463531941701, Read_Duration=6, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463531941578, firstTasksToStart=[ task_1463531941312_0001_1_01_000000 ], lastTaskFinishTime=1463531941786, lastTasksToFinish=[ task_1463531941312_0001_1_01_000000 ], minTaskDuration=208, maxTaskDuration=208, avgTaskDuration=208.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531941312_0001_1_01_000000 ], longestDurationTasks=[ task_1463531941312_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:01,794 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:01,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531941312_0001_00_000001, stoppedTime=1463531941794, exitStatus=0
2016-05-18 00:39:01,795 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] attempt: attempt_1463531941312_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:01,795 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:01,795 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:01,796 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531941312_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:01,800 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531941312_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:01,801 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:01,801 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531941312_0001_00_000001, Exiting
2016-05-18 00:39:01,801 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531941312_0001_00_000001
2016-05-18 00:39:01,801 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531941312_0001_00_000001
2016-05-18 00:39:01,803 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531941312_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:01,803 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531941312_0001_00_000001 completed successfully
2016-05-18 00:39:01,830 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,830 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEE5AEA24D5D42C89B76A913A579A1B3, mem on close (mb), free: 61, total: 498, max: 672
2016-05-18 00:39:01,831 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F470605B72BF427BB63906D14BCEFB71: Starting flush of map output
2016-05-18 00:39:01,831 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F470605B72BF427BB63906D14BCEFB71: Span0.length = 200, perItem = 45
2016-05-18 00:39:01,845 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F470605B72BF427BB63906D14BCEFB71: done sorting span=0, length=200, time=14
2016-05-18 00:39:01,845 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F470605B72BF427BB63906D14BCEFB71: Heap = SpanIterator<0:199> (span=Span[16777216,9061]),
2016-05-18 00:39:01,846 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F470605B72BF427BB63906D14BCEFB71: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-local-dir/output/attempt_1463531941312_0001_1_00_000000_0_10016_0/file.out
2016-05-18 00:39:01,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:01 Completed running task attempt: attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,854 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:01,854 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531941312_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:01,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531941312_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531941781, Process_Duration=49, Process_End_Time=1463531941830, Read_Duration=1, Tuples_Read=200, Tuples_Written=200, Write_Duration=21][cascading.flow.StepCounters Tuples_Read=200]]
2016-05-18 00:39:01,856 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531941312_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,856 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,856 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:01,856 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, taskAttemptId=attempt_1463531941312_0001_1_00_000000_0, creationTime=1463531941565, allocationTime=1463531941573, startTime=1463531941582, finishTime=1463531941857, timeTaken=275, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:01,857 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531941312_0001_1_00_000000_0
2016-05-18 00:39:01,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_FINISHED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, taskId=task_1463531941312_0001_1_00_000000, startTime=1463531941582, finishTime=1463531941858, timeTaken=276, status=SUCCEEDED, successfulAttemptID=attempt_1463531941312_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531941781, Process_Duration=49, Process_End_Time=1463531941830, Read_Duration=1, Tuples_Read=200, Tuples_Written=200, Write_Duration=21, cascading.flow.StepCounters, Tuples_Read=200
2016-05-18 00:39:01,859 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:01,859 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_FINISHED]: vertexName=EEE5AEA24D5D42C89B76A913A579A1B3, vertexId=vertex_1463531941312_0001_1_00, initRequestedTime=1463531941506, initedTime=1463531941549, startRequestedTime=1463531941508, startedTime=1463531941554, finishTime=1463531941859, timeTaken=305, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531941781, Process_Duration=49, Process_End_Time=1463531941830, Read_Duration=1, Tuples_Read=200, Tuples_Written=200, Write_Duration=21, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1463531941582, firstTasksToStart=[ task_1463531941312_0001_1_00_000000 ], lastTaskFinishTime=1463531941857, lastTasksToFinish=[ task_1463531941312_0001_1_00_000000 ], minTaskDuration=275, maxTaskDuration=275, avgTaskDuration=275.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531941312_0001_1_00_000000 ], longestDurationTasks=[ task_1463531941312_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:01,861 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:01,861 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] attempt: attempt_1463531941312_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:01,861 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F470605B72BF427BB63906D14BCEFB71
2016-05-18 00:39:01,859 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000002
2016-05-18 00:39:01,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531941312_0001_00_000002, stoppedTime=1463531941861, exitStatus=0
2016-05-18 00:39:01,866 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:01,862 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=EEE5AEA24D5D42C89B76A913A579A1B3, destinationVertexName=F470605B72BF427BB63906D14BCEFB71. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:01,866 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=EE038F2E8E584D58B0DEBBDDC84D453D, destinationVertexName=F470605B72BF427BB63906D14BCEFB71. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:01,867 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,866 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:01,869 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531941312_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:01,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_STARTED]: vertexName=F470605B72BF427BB63906D14BCEFB71, taskId=task_1463531941312_0001_1_02_000000, scheduledTime=1463531941869, launchTime=1463531941869
2016-05-18 00:39:01,869 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531941312_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:01,870 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,871 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:01,871 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531941312_0001_00_000002, Exiting
2016-05-18 00:39:01,871 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531941312_0001_00_000002
2016-05-18 00:39:01,871 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531941312_0001_00_000002
2016-05-18 00:39:01,873 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531941312_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:01,873 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531941312_0001_00_000002 completed successfully
2016-05-18 00:39:01,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531941312_0001_00_000003, launchTime=1463531941873
2016-05-18 00:39:01,874 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000003
2016-05-18 00:39:01,875 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531941312_0001_1_02_000000_0] started. Is using containerId: [container_1463531941312_0001_00_000003] on NM: [127.0.0.1:0]
2016-05-18 00:39:01,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F470605B72BF427BB63906D14BCEFB71, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0, startTime=1463531941874, containerId=container_1463531941312_0001_00_000003, nodeId=127.0.0.1:0
2016-05-18 00:39:01,876 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01 Starting to run new task attempt: attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,876 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,877 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:01,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: F470605B72BF427BB63906D14BCEFB71, VertexParallelism: 1, TaskAttemptID:attempt_1463531941312_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEE5AEA24D5D42C89B76A913A579A1B3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=EE038F2E8E584D58B0DEBBDDC84D453D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=89D7CE821C8640B0B2D2430CCBEDEF1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:01,877 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:01,878 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,880 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531941312_0001_1_02_000000_0 2 events [0,2) total 2 vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,881 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531941312_0001_1_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2
2016-05-18 00:39:01,883 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:01,886 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:01,888 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:01,893 INFO output.MROutput (MROutput.java:initialize(421)) - 89D7CE821C8640B0B2D2430CCBEDEF1E: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:01,902 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F470605B72BF427BB63906D14BCEFB71, ordinal: 2
2016-05-18 00:39:01,902 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F470605B72BF427BB63906D14BCEFB71, mem on start (mb), free: 54, total: 498, max: 672
2016-05-18 00:39:01,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:01,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 3 initializers to finish
2016-05-18 00:39:01,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:01,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:01,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:01,903 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[89D7CE821C8640B0B2D2430CCBEDEF1E:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [EEE5AEA24D5D42C89B76A913A579A1B3:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [EE038F2E8E584D58B0DEBBDDC84D453D:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:01,903 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:01,904 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-05-18 00:39:01,904 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 422785824, TotalRequestedScaled: 2.0293719551999998E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:0.60
2016-05-18 00:39:01,904 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[89D7CE821C8640B0B2D2430CCBEDEF1E:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [EEE5AEA24D5D42C89B76A913A579A1B3:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [EE038F2E8E584D58B0DEBBDDC84D453D:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:01,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:01,905 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - EEE5AEA24D5D42C89B76A913A579A1B3: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:01,905 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - EEE5AEA24D5D42C89B76A913A579A1B3: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:01,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: EEE5AEA24D5D42C89B76A913A579A1B3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:01,907 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: EEE5AEA24D5D42C89B76A913A579A1B3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:01,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: EE038F2E8E584D58B0DEBBDDC84D453D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:01,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: EEE5AEA24D5D42C89B76A913A579A1B3
2016-05-18 00:39:01,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - EE038F2E8E584D58B0DEBBDDC84D453D: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:01,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 2
2016-05-18 00:39:01,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 2 IOs to start
2016-05-18 00:39:01,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:01,912 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - EE038F2E8E584D58B0DEBBDDC84D453D: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:01,913 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: EE038F2E8E584D58B0DEBBDDC84D453D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:01,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: EE038F2E8E584D58B0DEBBDDC84D453D
2016-05-18 00:39:01,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:01,917 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,917 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - EE038F2E8E584D58B0DEBBDDC84D453D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:01,918 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - EEE5AEA24D5D42C89B76A913A579A1B3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:01,922 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531941312_0001_1_00_000000_0_10016} to DISK_DIRECT, csize=8931, dsize=8927, EndTime=1463531941921, TimeTaken=2, Rate=4.26 MB/s
2016-05-18 00:39:01,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : EEE5AEA24D5D42C89B76A913A579A1B3
2016-05-18 00:39:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-05-18 00:39:01,922 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531941312_0001_1_01_000000_0_10012} to DISK_DIRECT, csize=142, dsize=138, EndTime=1463531941922, TimeTaken=2, Rate=0.07 MB/s
2016-05-18 00:39:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: EEE5AEA24D5D42C89B76A913A579A1B3, wasInterrupted=false
2016-05-18 00:39:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : EE038F2E8E584D58B0DEBBDDC84D453D
2016-05-18 00:39:01,925 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:01,925 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: EE038F2E8E584D58B0DEBBDDC84D453D, wasInterrupted=false
2016-05-18 00:39:01,925 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-05-18 00:39:01,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - EE038F2E8E584D58B0DEBBDDC84D453D: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:01,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:01,928 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:01,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-local-dir/output/attempt_1463531941312_0001_1_00_000000_0_10016/file.out, len=8931, isLocal=true
2016-05-18 00:39:01,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 8931 bytes from disk
2016-05-18 00:39:01,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:01,930 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:01,930 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 8918 bytes
2016-05-18 00:39:01,931 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : EEE5AEA24D5D42C89B76A913A579A1B3
2016-05-18 00:39:01,931 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - EEE5AEA24D5D42C89B76A913A579A1B3: Shuffle Runner thread complete
2016-05-18 00:39:01,931 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:01,932 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001_wd/localmode-local-dir/output/attempt_1463531941312_0001_1_01_000000_0_10012/file.out, len=142, isLocal=true
2016-05-18 00:39:01,932 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 142 bytes from disk
2016-05-18 00:39:01,933 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:01,933 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:01,933 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 129 bytes
2016-05-18 00:39:01,933 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : EE038F2E8E584D58B0DEBBDDC84D453D
2016-05-18 00:39:01,933 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - EE038F2E8E584D58B0DEBBDDC84D453D: Shuffle Runner thread complete
2016-05-18 00:39:01,935 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-05-18 00:39:01,936 WARN util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@6bf561ca, and: cascading.platform.hadoop.TestLongComparator@3ae74c25
2016-05-18 00:39:01,936 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: DA28085F4246455196F9F400F6B7631A
2016-05-18 00:39:01,937 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"], id: 89D7CE821C8640B0B2D2430CCBEDEF1E
2016-05-18 00:39:01,937 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-05-18 00:39:01,938 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] DA28085F4246455196F9F400F6B7631A, for 2 inputs
2016-05-18 00:39:01,938 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] DA28085F4246455196F9F400F6B7631A, for 2 inputs
2016-05-18 00:39:01,939 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 00:39:01,939 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-05-18 00:39:01,940 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F470605B72BF427BB63906D14BCEFB71, all 2 inputs ready in: 00:00:00.000
2016-05-18 00:39:01,944 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - EEE5AEA24D5D42C89B76A913A579A1B3: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:01,946 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - EE038F2E8E584D58B0DEBBDDC84D453D: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:01,980 INFO output.MROutput (MROutput.java:close(526)) - 89D7CE821C8640B0B2D2430CCBEDEF1E closed
2016-05-18 00:39:01,981 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531941312_0001_1_02_000000_0 given a go for committing the task output.
2016-05-18 00:39:01,982 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,983 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F470605B72BF427BB63906D14BCEFB71, mem on close (mb), free: 49, total: 498, max: 672
2016-05-18 00:39:01,983 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: EEE5AEA24D5D42C89B76A913A579A1B3
2016-05-18 00:39:01,983 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - EEE5AEA24D5D42C89B76A913A579A1B3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:01,984 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: EE038F2E8E584D58B0DEBBDDC84D453D
2016-05-18 00:39:01,985 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - EE038F2E8E584D58B0DEBBDDC84D453D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:01,985 INFO output.MROutput (MROutput.java:close(526)) - 89D7CE821C8640B0B2D2430CCBEDEF1E closed
2016-05-18 00:39:01,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:01 Completed running task attempt: attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,987 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0, askedToStop=false
2016-05-18 00:39:01,987 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531941312_0001_1_02_000000_0, stopRequested=false
2016-05-18 00:39:01,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531941312_0001_1_02_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=48, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531941941, Process_Duration=41, Process_End_Time=1463531941982, Read_Duration=8, Tuples_Read=124, Tuples_Written=199, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=199]]
2016-05-18 00:39:01,990 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531941312_0001_1_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,990 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:01,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F470605B72BF427BB63906D14BCEFB71, taskAttemptId=attempt_1463531941312_0001_1_02_000000_0, creationTime=1463531941870, allocationTime=1463531941871, startTime=1463531941874, finishTime=1463531941990, timeTaken=116, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:01,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:01,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:01,991 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531941312_0001_1_02_000000_0
2016-05-18 00:39:01,992 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531941312_0001_00_000003
2016-05-18 00:39:01,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:TASK_FINISHED]: vertexName=F470605B72BF427BB63906D14BCEFB71, taskId=task_1463531941312_0001_1_02_000000, startTime=1463531941874, finishTime=1463531941991, timeTaken=117, status=SUCCEEDED, successfulAttemptID=attempt_1463531941312_0001_1_02_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=48, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531941941, Process_Duration=41, Process_End_Time=1463531941982, Read_Duration=8, Tuples_Read=124, Tuples_Written=199, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=199
2016-05-18 00:39:01,993 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:01,993 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71]
2016-05-18 00:39:01,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:VERTEX_FINISHED]: vertexName=F470605B72BF427BB63906D14BCEFB71, vertexId=vertex_1463531941312_0001_1_02, initRequestedTime=1463531941508, initedTime=1463531941521, startRequestedTime=1463531941557, startedTime=1463531941560, finishTime=1463531941993, timeTaken=433, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=48, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531941941, Process_Duration=41, Process_End_Time=1463531941982, Read_Duration=8, Tuples_Read=124, Tuples_Written=199, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1463531941874, firstTasksToStart=[ task_1463531941312_0001_1_02_000000 ], lastTaskFinishTime=1463531941990, lastTasksToFinish=[ task_1463531941312_0001_1_02_000000 ], minTaskDuration=116, maxTaskDuration=116, avgTaskDuration=116.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531941312_0001_1_02_000000 ], longestDurationTasks=[ task_1463531941312_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:01,995 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:01,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531941312_0001_00_000003, stoppedTime=1463531941995, exitStatus=0
2016-05-18 00:39:01,996 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-05-18 00:39:01,996 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:01,997 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531941312_0001_1_01 [EE038F2E8E584D58B0DEBBDDC84D453D]
2016-05-18 00:39:01,997 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531941312_0001_1_00 [EEE5AEA24D5D42C89B76A913A579A1B3]
2016-05-18 00:39:01,997 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531941312_0001_1
2016-05-18 00:39:01,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531941312_0001_1
2016-05-18 00:39:01,997 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531941312_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:01,997 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 89D7CE821C8640B0B2D2430CCBEDEF1E for vertex: vertex_1463531941312_0001_1_02 [F470605B72BF427BB63906D14BCEFB71], outputName: 89D7CE821C8640B0B2D2430CCBEDEF1E
2016-05-18 00:39:01,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531941312_0001_00_000003. Relying on regular task shutdown for it to end
2016-05-18 00:39:02,002 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531941312_0001_00_000003 is valid, but no longer registered, and will be killed
2016-05-18 00:39:02,002 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531941312_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:02,003 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531941312_0001_00_000003, Exiting
2016-05-18 00:39:02,003 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531941312_0001_00_000003
2016-05-18 00:39:02,003 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531941312_0001_00_000003
2016-05-18 00:39:02,003 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531941312_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:02,003 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531941312_0001_00_000003 completed successfully
2016-05-18 00:39:02,007 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:89D7CE821C8640B0B2D2430CCBEDEF1E of vertex/vertexGroup:F470605B72BF427BB63906D14BCEFB71 isVertexGroupOutput:false
2016-05-18 00:39:02,007 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null
2016-05-18 00:39:02,011 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531941312_0001_1, queueSize=0
2016-05-18 00:39:02,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531941312_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531941312_0001_1, startTime=1463531941504, finishTime=1463531942007, timeTaken=503, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=1060, AM_GC_TIME_MILLIS=22, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=69, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=32, MERGE_PHASE_TIME=48, FIRST_EVENT_RECEIVED=19, LAST_EVENT_RECEIVED=19, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4390595825414, Process_Duration=99, Process_End_Time=4390595825513, Read_Duration=15, Tuples_Read=344, Tuples_Written=419, Write_Duration=23, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-05-18 00:39:02,012 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531941312_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:02,012 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531941312_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:02 Completed Dag: dag_1463531941312_0001_1
2016-05-18 00:39:02,014 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531941312_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:02,014 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531941312_0001_1_post
2016-05-18 00:39:02,015 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:02,016 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:02,016 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:02,016 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward, with id=dag_1463531941312_0001_1
2016-05-18 00:39:02,017 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1] apache*ip/(1/1) ...ues/cogroupcompareforward, with id=dag_1463531941312_0001_1
2016-05-18 00:39:02,023 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1463531941312_0001
2016-05-18 00:39:02,030 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:02,030 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:02,030 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:02,031 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:02,031 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:02,031 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:02,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:02,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:02,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:02,034 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:02,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:02,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:02,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:02,036 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:02,039 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/ABE4625436184E7D9EE85DCB5CAD221C/B1EE611D01D14FDFB2CA639D89EE70B1/.tez/application_1463531941312_0001
2016-05-18 00:39:02,040 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward/_temporary
2016-05-18 00:39:02,041 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache*ip] completed in: 00:00.759
2016-05-18 00:39:02,557 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-05-18 00:39:02,592 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:02,592 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:02,635 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-18 00:39:02,639 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.013
2016-05-18 00:39:02,639 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:39:02,640 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [apache] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-18 00:39:02,671 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:02,671 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:02,729 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-18 00:39:02,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-05-18 00:39:02,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:02,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:02,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:02,761 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:02,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false
2016-05-18 00:39:02,844 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:02,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:02,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-05-18 00:39:02,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:02,847 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:02,847 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:02,848 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ve/false_null_false_false
2016-05-18 00:39:02,848 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:02,870 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:02,875 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:02,875 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:02,875 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:02,885 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001 doesn't exist and is created
2016-05-18 00:39:02,904 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:02,904 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:02,905 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd
2016-05-18 00:39:02,920 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531942875_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:02,920 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:02,920 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:02,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:02,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:02,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:02,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:02,925 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:02,927 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:02,927 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:02,927 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:02,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:02,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:02,928 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:02,928 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:02,929 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:02,929 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:02,929 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:02,929 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:02,930 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:02,932 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:02,933 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:02,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:02,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-log-dir
2016-05-18 00:39:02,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-log-dir/history.txt.appattempt_1463531942875_0001_000000, maxErrors=10
2016-05-18 00:39:02,934 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:02,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531942875_0001_000000, appSubmitTime=1463531942919, launchTime=1463531942919
2016-05-18 00:39:02,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:02,951 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34157
2016-05-18 00:39:02,952 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531942875
2016-05-18 00:39:02,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531942875_0001_000000, startTime=1463531942955
2016-05-18 00:39:02,955 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:03,004 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:03,005 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:03,005 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1463531942875_0001, dagName=[07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-05-18 00:39:03,027 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-05-18 00:39:03,028 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:03,029 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531942875_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-log-dir/dag_1463531942875_0001_1.dot
2016-05-18 00:39:03,030 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-log-dir/dag_1463531942875_0001_1-tez-dag.pb.txt
2016-05-18 00:39:03,038 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531942875_0001_1
2016-05-18 00:39:03,038 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,038 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-05-18 00:39:03 Running Dag: dag_1463531942875_0001_1
2016-05-18 00:39:03,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531942875_0001_1, submitTime=1463531943028
2016-05-18 00:39:03,050 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001/tez-conf.pb" } size: 36903 timestamp: 1463531942000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:03,050 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,050 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:03,051 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001/tez-conf.pb" } size: 36903 timestamp: 1463531942000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:03,051 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,051 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:03,051 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:03,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531942875_0001_1, initTime=1463531943052
2016-05-18 00:39:03,052 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531942875_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:03,053 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:03,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:DAG_STARTED]: dagID=dag_1463531942875_0001_1, startTime=1463531943053
2016-05-18 00:39:03,053 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531942875_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:03,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BF21AC65CAF94BB9B2745E45A2F2135C : {1EB3C67766DE491699A7F66B28E31572={InputName=1EB3C67766DE491699A7F66B28E31572}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:03,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1EB3C67766DE491699A7F66B28E31572, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:03,054 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,055 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,055 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:03,055 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,055 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1463531942875_0001, dagName=[07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-05-18 00:39:03,055 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1EB3C67766DE491699A7F66B28E31572 on vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,058 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1EB3C67766DE491699A7F66B28E31572 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:03,060 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:03,061 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,061 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,061 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:03,061 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:03,062 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:03,062 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:03,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531942875_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:03,064 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A], numAdditionalOutputs=1
2016-05-18 00:39:03,064 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F4C07E059EA3484FB848B640CE10FB43, vertex=vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:03,064 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1EB3C67766DE491699A7F66B28E31572 on vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,066 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CFC427ABA847481AA4F00E39C8FA766A:F4C07E059EA3484FB848B640CE10FB43 using oldmapred API
2016-05-18 00:39:03,067 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F4C07E059EA3484FB848B640CE10FB43, vertexName=CFC427ABA847481AA4F00E39C8FA766A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:03,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, vertexId=vertex_1463531942875_0001_1_01, initRequestedTime=1463531943055, initedTime=1463531943064, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:03,068 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:03,070 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:03,070 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1EB3C67766DE491699A7F66B28E31572=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:03,071 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] parallelism set to 1
2016-05-18 00:39:03,071 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:03,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, vertexId=vertex_1463531942875_0001_1_00, initRequestedTime=1463531943054, initedTime=1463531943072, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:03,072 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] with distanceFromRoot: 0
2016-05-18 00:39:03,073 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:03,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531942875_0001_1_00, startRequestedTime=1463531943055, startedTime=1463531943073
2016-05-18 00:39:03,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531942875_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:03,074 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:03,075 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BF21AC65CAF94BB9B2745E45A2F2135C
2016-05-18 00:39:03,075 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,075 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531942875_0001_1_00 for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] numStartedSources: 1 numSources: 1
2016-05-18 00:39:03,075 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] with distanceFromRoot: 1
2016-05-18 00:39:03,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531942875_0001_1_01, startRequestedTime=1463531943075, startedTime=1463531943075
2016-05-18 00:39:03,076 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:03,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_STARTED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, taskId=task_1463531942875_0001_1_00_000000, scheduledTime=1463531943076, launchTime=1463531943076
2016-05-18 00:39:03,077 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,079 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: CFC427ABA847481AA4F00E39C8FA766A with 0 source tasks and 1 pending tasks
2016-05-18 00:39:03,079 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:03,079 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: BF21AC65CAF94BB9B2745E45A2F2135C in vertex: CFC427ABA847481AA4F00E39C8FA766A numBipartiteSourceTasks: 1
2016-05-18 00:39:03,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531942875_0001_00_000001, launchTime=1463531943081
2016-05-18 00:39:03,083 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531942875_0001_00_000001
2016-05-18 00:39:03,084 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531942875_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,084 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531942875_0001_1_00_000000_0] started. Is using containerId: [container_1463531942875_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:03,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0, startTime=1463531943084, containerId=container_1463531942875_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:03 Starting to run new task attempt: attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,087 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,087 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,087 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:03,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: BF21AC65CAF94BB9B2745E45A2F2135C, VertexParallelism: 1, TaskAttemptID:attempt_1463531942875_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1EB3C67766DE491699A7F66B28E31572, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CFC427ABA847481AA4F00E39C8FA766A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:03,088 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:03,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531942875_0001, with dag id: dag_1463531942875_0001_1
2016-05-18 00:39:03,092 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,093 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531942875_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:03,094 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531942875_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:03,097 INFO input.MRInput (MRInput.java:initialize(440)) - 1EB3C67766DE491699A7F66B28E31572 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:03,098 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1EB3C67766DE491699A7F66B28E31572
2016-05-18 00:39:03,098 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:03,108 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF21AC65CAF94BB9B2745E45A2F2135C, ordinal: 0
2016-05-18 00:39:03,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF21AC65CAF94BB9B2745E45A2F2135C, mem on start (mb), free: 228, total: 351, max: 672
2016-05-18 00:39:03,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:03,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:03,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:03,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:03,109 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1EB3C67766DE491699A7F66B28E31572:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CFC427ABA847481AA4F00E39C8FA766A:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:03,109 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:03,110 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:03,110 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:03,110 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1EB3C67766DE491699A7F66B28E31572:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CFC427ABA847481AA4F00E39C8FA766A:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:03,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:03,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1EB3C67766DE491699A7F66B28E31572 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:03,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:03,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:03,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1EB3C67766DE491699A7F66B28E31572
2016-05-18 00:39:03,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:03,112 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,114 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:03,116 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 1EB3C67766DE491699A7F66B28E31572
2016-05-18 00:39:03,116 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:03,116 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1EB3C67766DE491699A7F66B28E31572 initialized RecordReader from event
2016-05-18 00:39:03,116 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: BC6F244B01114845AEF21812C42323A8
2016-05-18 00:39:03,117 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] BC6F244B01114845AEF21812C42323A8
2016-05-18 00:39:03,121 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - CFC427ABA847481AA4F00E39C8FA766A using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@293d6af1, comparator=cascading.tuple.hadoop.util.TupleComparator@7f38095d, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:03,121 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for CFC427ABA847481AA4F00E39C8FA766A: , UsingHashComparator=false
2016-05-18 00:39:03,136 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:03,137 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:03,137 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for CFC427ABA847481AA4F00E39C8FA766A: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:03,137 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - CFC427ABA847481AA4F00E39C8FA766A: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:03,138 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:03,138 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF21AC65CAF94BB9B2745E45A2F2135C, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:03,143 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,143 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF21AC65CAF94BB9B2745E45A2F2135C, mem on close (mb), free: 126, total: 351, max: 672
2016-05-18 00:39:03,144 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - CFC427ABA847481AA4F00E39C8FA766A: Starting flush of map output
2016-05-18 00:39:03,144 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - CFC427ABA847481AA4F00E39C8FA766A: Span0.length = 37, perItem = 17
2016-05-18 00:39:03,147 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - CFC427ABA847481AA4F00E39C8FA766A: done sorting span=0, length=37, time=3
2016-05-18 00:39:03,147 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - CFC427ABA847481AA4F00E39C8FA766A: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:03,148 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - CFC427ABA847481AA4F00E39C8FA766A: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-local-dir/output/attempt_1463531942875_0001_1_00_000000_0_10023_0/file.out
2016-05-18 00:39:03,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:03 Completed running task attempt: attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,153 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:03,153 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531942875_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:03,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531942875_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531943138, Process_Duration=5, Process_End_Time=1463531943143, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:03,155 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531942875_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,155 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,155 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:03,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, taskAttemptId=attempt_1463531942875_0001_1_00_000000_0, creationTime=1463531943077, allocationTime=1463531943081, startTime=1463531943084, finishTime=1463531943155, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:03,156 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531942875_0001_1_00_000000_0
2016-05-18 00:39:03,157 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,157 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531942875_0001_00_000001
2016-05-18 00:39:03,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_FINISHED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, taskId=task_1463531942875_0001_1_00_000000, startTime=1463531943084, finishTime=1463531943156, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463531942875_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531943138, Process_Duration=5, Process_End_Time=1463531943143, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:03,158 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:03,158 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_FINISHED]: vertexName=BF21AC65CAF94BB9B2745E45A2F2135C, vertexId=vertex_1463531942875_0001_1_00, initRequestedTime=1463531943054, initedTime=1463531943072, startRequestedTime=1463531943055, startedTime=1463531943073, finishTime=1463531943158, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531943138, Process_Duration=5, Process_End_Time=1463531943143, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531943084, firstTasksToStart=[ task_1463531942875_0001_1_00_000000 ], lastTaskFinishTime=1463531943155, lastTasksToFinish=[ task_1463531942875_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531942875_0001_1_00_000000 ], longestDurationTasks=[ task_1463531942875_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:03,160 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:03,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531942875_0001_00_000001, stoppedTime=1463531943160, exitStatus=0
2016-05-18 00:39:03,160 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] attempt: attempt_1463531942875_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:03,161 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:03,161 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CFC427ABA847481AA4F00E39C8FA766A
2016-05-18 00:39:03,161 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=BF21AC65CAF94BB9B2745E45A2F2135C, destinationVertexName=CFC427ABA847481AA4F00E39C8FA766A. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:03,161 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,162 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,162 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531942875_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:03,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_STARTED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, taskId=task_1463531942875_0001_1_01_000000, scheduledTime=1463531943162, launchTime=1463531943162
2016-05-18 00:39:03,163 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531942875_0001_00_000002, launchTime=1463531943166
2016-05-18 00:39:03,166 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531942875_0001_00_000002
2016-05-18 00:39:03,167 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531942875_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,167 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531942875_0001_1_01_000000_0] started. Is using containerId: [container_1463531942875_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:03 Starting to run new task attempt: attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,168 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,168 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,168 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:03,168 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531942875_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:03,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0, startTime=1463531943167, containerId=container_1463531942875_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:03,168 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531942875_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:03,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: CFC427ABA847481AA4F00E39C8FA766A, VertexParallelism: 1, TaskAttemptID:attempt_1463531942875_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BF21AC65CAF94BB9B2745E45A2F2135C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F4C07E059EA3484FB848B640CE10FB43, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:03,168 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531942875_0001_00_000001, Exiting
2016-05-18 00:39:03,169 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531942875_0001_00_000001
2016-05-18 00:39:03,169 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:03,169 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531942875_0001_00_000001
2016-05-18 00:39:03,170 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,171 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531942875_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531942875_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:03,171 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531942875_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:03,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531942875_0001_00_000001 completed successfully
2016-05-18 00:39:03,176 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:03,176 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:03,183 INFO output.MROutput (MROutput.java:initialize(421)) - F4C07E059EA3484FB848B640CE10FB43: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:03,187 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CFC427ABA847481AA4F00E39C8FA766A, ordinal: 1
2016-05-18 00:39:03,187 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFC427ABA847481AA4F00E39C8FA766A, mem on start (mb), free: 121, total: 351, max: 672
2016-05-18 00:39:03,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:03,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:03,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:03,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:03,188 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F4C07E059EA3484FB848B640CE10FB43:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [BF21AC65CAF94BB9B2745E45A2F2135C:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:03,188 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:03,188 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:03,188 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:03,189 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F4C07E059EA3484FB848B640CE10FB43:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [BF21AC65CAF94BB9B2745E45A2F2135C:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:03,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:03,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BF21AC65CAF94BB9B2745E45A2F2135C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:03,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:03,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:03,190 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - BF21AC65CAF94BB9B2745E45A2F2135C: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:03,190 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - BF21AC65CAF94BB9B2745E45A2F2135C: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:03,191 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: BF21AC65CAF94BB9B2745E45A2F2135C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:03,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BF21AC65CAF94BB9B2745E45A2F2135C
2016-05-18 00:39:03,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:03,193 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,194 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BF21AC65CAF94BB9B2745E45A2F2135C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:03,198 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531942875_0001_1_00_000000_0_10023} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531943198, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:03,198 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:03,198 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : BF21AC65CAF94BB9B2745E45A2F2135C
2016-05-18 00:39:03,199 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: BC6F244B01114845AEF21812C42323A8
2016-05-18 00:39:03,199 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:03,199 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"], id: F4C07E059EA3484FB848B640CE10FB43
2016-05-18 00:39:03,199 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-05-18 00:39:03,200 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] BC6F244B01114845AEF21812C42323A8, for 1 inputs
2016-05-18 00:39:03,200 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: BF21AC65CAF94BB9B2745E45A2F2135C, wasInterrupted=false
2016-05-18 00:39:03,200 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:03,201 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:03,202 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001_wd/localmode-local-dir/output/attempt_1463531942875_0001_1_00_000000_0_10023/file.out, len=593, isLocal=true
2016-05-18 00:39:03,202 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:03,202 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:03,202 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:03,203 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:03,203 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CFC427ABA847481AA4F00E39C8FA766A, all 1 inputs ready in: 00:00:00.003
2016-05-18 00:39:03,203 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : BF21AC65CAF94BB9B2745E45A2F2135C
2016-05-18 00:39:03,204 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - BF21AC65CAF94BB9B2745E45A2F2135C: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:03,205 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - BF21AC65CAF94BB9B2745E45A2F2135C: Shuffle Runner thread complete
2016-05-18 00:39:03,220 INFO output.MROutput (MROutput.java:close(526)) - F4C07E059EA3484FB848B640CE10FB43 closed
2016-05-18 00:39:03,221 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531942875_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:03,222 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,222 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CFC427ABA847481AA4F00E39C8FA766A, mem on close (mb), free: 117, total: 351, max: 672
2016-05-18 00:39:03,222 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: BF21AC65CAF94BB9B2745E45A2F2135C
2016-05-18 00:39:03,222 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BF21AC65CAF94BB9B2745E45A2F2135C: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:03,223 INFO output.MROutput (MROutput.java:close(526)) - F4C07E059EA3484FB848B640CE10FB43 closed
2016-05-18 00:39:03,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:03 Completed running task attempt: attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,223 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:03,224 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531942875_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:03,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531942875_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531943203, Process_Duration=19, Process_End_Time=1463531943222, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:03,225 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531942875_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,225 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,225 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:03,226 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,226 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531942875_0001_00_000002
2016-05-18 00:39:03,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, taskAttemptId=attempt_1463531942875_0001_1_01_000000_0, creationTime=1463531943163, allocationTime=1463531943165, startTime=1463531943167, finishTime=1463531943225, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:03,226 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531942875_0001_1_01_000000_0
2016-05-18 00:39:03,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:TASK_FINISHED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, taskId=task_1463531942875_0001_1_01_000000, startTime=1463531943167, finishTime=1463531943227, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463531942875_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531943203, Process_Duration=19, Process_End_Time=1463531943222, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:03,228 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:03,228 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A]
2016-05-18 00:39:03,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:VERTEX_FINISHED]: vertexName=CFC427ABA847481AA4F00E39C8FA766A, vertexId=vertex_1463531942875_0001_1_01, initRequestedTime=1463531943055, initedTime=1463531943064, startRequestedTime=1463531943075, startedTime=1463531943075, finishTime=1463531943228, timeTaken=153, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531943203, Process_Duration=19, Process_End_Time=1463531943222, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531943167, firstTasksToStart=[ task_1463531942875_0001_1_01_000000 ], lastTaskFinishTime=1463531943225, lastTasksToFinish=[ task_1463531942875_0001_1_01_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531942875_0001_1_01_000000 ], longestDurationTasks=[ task_1463531942875_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:03,230 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:03,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531942875_0001_00_000002, stoppedTime=1463531943233, exitStatus=0
2016-05-18 00:39:03,233 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:03,233 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,233 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531942875_0001_1_00 [BF21AC65CAF94BB9B2745E45A2F2135C]
2016-05-18 00:39:03,234 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531942875_0001_1
2016-05-18 00:39:03,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531942875_0001_1
2016-05-18 00:39:03,234 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F4C07E059EA3484FB848B640CE10FB43 for vertex: vertex_1463531942875_0001_1_01 [CFC427ABA847481AA4F00E39C8FA766A], outputName: F4C07E059EA3484FB848B640CE10FB43
2016-05-18 00:39:03,236 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531942875_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:03,236 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531942875_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:03,237 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531942875_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:03,237 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531942875_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:03,238 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531942875_0001_00_000002, Exiting
2016-05-18 00:39:03,238 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531942875_0001_00_000002
2016-05-18 00:39:03,238 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531942875_0001_00_000002
2016-05-18 00:39:03,238 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531942875_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:03,239 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531942875_0001_00_000002 completed successfully
2016-05-18 00:39:03,243 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F4C07E059EA3484FB848B640CE10FB43 of vertex/vertexGroup:CFC427ABA847481AA4F00E39C8FA766A isVertexGroupOutput:false
2016-05-18 00:39:03,243 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,246 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531942875_0001_1, queueSize=0
2016-05-18 00:39:03,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531942875_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531942875_0001_1, startTime=1463531943053, finishTime=1463531943243, timeTaken=190, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=350, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063886341, Process_Duration=24, Process_End_Time=2927063886365, Read_Duration=5, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:03,246 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531942875_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:03,247 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531942875_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:03 Completed Dag: dag_1463531942875_0001_1
2016-05-18 00:39:03,248 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531942875_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:03,249 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531942875_0001_1_post
2016-05-18 00:39:03,250 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:03,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:03,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false, with id=dag_1463531942875_0001_1
2016-05-18 00:39:03,250 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131] comprehensivesort/(1/1) ...ve/false_null_false_false, with id=dag_1463531942875_0001_1
2016-05-18 00:39:03,254 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1463531942875_0001
2016-05-18 00:39:03,256 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:03,256 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:03,256 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:03,257 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:03,257 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:03,257 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:03,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:03,258 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:03,258 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:03,258 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:03,258 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:03,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:03,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:03,261 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:03,266 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/07E84D1397CD453187266F676AEBAA0F/ACA560F5A46D44BB9F6D3676D581F131/.tez/application_1463531942875_0001
2016-05-18 00:39:03,266 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false/_temporary
2016-05-18 00:39:03,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.422
2016-05-18 00:39:03,358 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:03,358 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:03,414 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 00:39:03,422 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-18 00:39:03,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:03,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:03,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:03,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:03,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:03,520 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:03,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/true_null_false_false
2016-05-18 00:39:03,521 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:03,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:03,541 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:03,541 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:03,541 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:03,550 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001 doesn't exist and is created
2016-05-18 00:39:03,564 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:03,564 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:03,565 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd
2016-05-18 00:39:03,576 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531943541_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:03,576 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:03,576 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:03,578 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:03,578 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:03,578 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:03,578 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:03,579 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:03,580 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:03,580 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:03,580 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:03,580 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:03,580 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:03,581 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:03,581 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:03,581 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:03,581 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:03,581 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:03,581 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:03,581 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:03,583 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:03,586 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:03,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:03,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-log-dir
2016-05-18 00:39:03,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-log-dir/history.txt.appattempt_1463531943541_0001_000000, maxErrors=10
2016-05-18 00:39:03,587 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:03,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531943541_0001_000000, appSubmitTime=1463531943575, launchTime=1463531943575
2016-05-18 00:39:03,598 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35759
2016-05-18 00:39:03,598 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531943541
2016-05-18 00:39:03,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:03,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531943541_0001_000000, startTime=1463531943601
2016-05-18 00:39:03,601 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:03,664 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:03,665 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:03,665 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1463531943541_0001, dagName=[CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-05-18 00:39:03,684 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-05-18 00:39:03,684 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:03,685 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531943541_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-log-dir/dag_1463531943541_0001_1.dot
2016-05-18 00:39:03,686 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-log-dir/dag_1463531943541_0001_1-tez-dag.pb.txt
2016-05-18 00:39:03,693 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531943541_0001_1
2016-05-18 00:39:03,693 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,693 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-05-18 00:39:03 Running Dag: dag_1463531943541_0001_1
2016-05-18 00:39:03,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531943541_0001_1, submitTime=1463531943684
2016-05-18 00:39:03,705 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001/tez-conf.pb" } size: 36903 timestamp: 1463531943000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:03,706 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,706 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:03,708 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001/tez-conf.pb" } size: 36903 timestamp: 1463531943000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:03,708 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,708 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:03,709 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:03,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531943541_0001_1, initTime=1463531943710
2016-05-18 00:39:03,711 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531943541_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:03,712 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:03,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:DAG_STARTED]: dagID=dag_1463531943541_0001_1, startTime=1463531943712
2016-05-18 00:39:03,712 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531943541_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C92F0EFF4C7C412E905C988861769BB0 : {60DCD32114994EF7B753A4BBDF74F75E={InputName=60DCD32114994EF7B753A4BBDF74F75E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 60DCD32114994EF7B753A4BBDF74F75E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,713 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,714 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1463531943541_0001, dagName=[CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-05-18 00:39:03,717 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:03,717 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,719 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 60DCD32114994EF7B753A4BBDF74F75E on vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,721 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:03,721 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,722 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531943541_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:03,724 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA], numAdditionalOutputs=1
2016-05-18 00:39:03,724 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=CC847EB21BA546CD84F8A2B3BE978D48, vertex=vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:03,724 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 60DCD32114994EF7B753A4BBDF74F75E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:03,728 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E6BFBFADE4F2422B812F99A77DDFAAAA:CC847EB21BA546CD84F8A2B3BE978D48 using oldmapred API
2016-05-18 00:39:03,728 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=CC847EB21BA546CD84F8A2B3BE978D48, vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:03,728 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:03,729 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:03,730 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:03,730 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:03,731 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 60DCD32114994EF7B753A4BBDF74F75E on vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, vertexId=vertex_1463531943541_0001_1_01, initRequestedTime=1463531943717, initedTime=1463531943724, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:03,732 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:03,734 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:03,737 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {60DCD32114994EF7B753A4BBDF74F75E=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:03,738 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] parallelism set to 1
2016-05-18 00:39:03,738 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:03,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, vertexId=vertex_1463531943541_0001_1_00, initRequestedTime=1463531943712, initedTime=1463531943739, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:03,739 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] with distanceFromRoot: 0
2016-05-18 00:39:03,739 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:03,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531943541_0001_1_00, startRequestedTime=1463531943717, startedTime=1463531943740
2016-05-18 00:39:03,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531943541_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:03,740 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:03,740 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531943541_0001_1_00 for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] numStartedSources: 1 numSources: 1
2016-05-18 00:39:03,740 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] with distanceFromRoot: 1
2016-05-18 00:39:03,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531943541_0001_1_01, startRequestedTime=1463531943740, startedTime=1463531943741
2016-05-18 00:39:03,744 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:03,750 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C92F0EFF4C7C412E905C988861769BB0
2016-05-18 00:39:03,750 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_STARTED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, taskId=task_1463531943541_0001_1_00_000000, scheduledTime=1463531943750, launchTime=1463531943750
2016-05-18 00:39:03,751 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: E6BFBFADE4F2422B812F99A77DDFAAAA with 0 source tasks and 1 pending tasks
2016-05-18 00:39:03,751 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531943541_0001, with dag id: dag_1463531943541_0001_1
2016-05-18 00:39:03,753 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: C92F0EFF4C7C412E905C988861769BB0 in vertex: E6BFBFADE4F2422B812F99A77DDFAAAA numBipartiteSourceTasks: 1
2016-05-18 00:39:03,755 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:03,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531943541_0001_00_000001, launchTime=1463531943758
2016-05-18 00:39:03,759 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531943541_0001_00_000001
2016-05-18 00:39:03,760 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531943541_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,761 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531943541_0001_1_00_000000_0] started. Is using containerId: [container_1463531943541_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:03,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0, startTime=1463531943760, containerId=container_1463531943541_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:03 Starting to run new task attempt: attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,762 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,763 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:03,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: C92F0EFF4C7C412E905C988861769BB0, VertexParallelism: 1, TaskAttemptID:attempt_1463531943541_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=60DCD32114994EF7B753A4BBDF74F75E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:03,764 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:03,764 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,766 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531943541_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:03,767 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531943541_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:03,770 INFO input.MRInput (MRInput.java:initialize(440)) - 60DCD32114994EF7B753A4BBDF74F75E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:03,770 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 60DCD32114994EF7B753A4BBDF74F75E
2016-05-18 00:39:03,771 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:03,778 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C92F0EFF4C7C412E905C988861769BB0, ordinal: 0
2016-05-18 00:39:03,778 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C92F0EFF4C7C412E905C988861769BB0, mem on start (mb), free: 218, total: 327, max: 672
2016-05-18 00:39:03,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:03,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:03,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:03,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:03,779 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[60DCD32114994EF7B753A4BBDF74F75E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [E6BFBFADE4F2422B812F99A77DDFAAAA:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:03,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:03,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:03,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:03,779 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[60DCD32114994EF7B753A4BBDF74F75E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [E6BFBFADE4F2422B812F99A77DDFAAAA:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 60DCD32114994EF7B753A4BBDF74F75E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 60DCD32114994EF7B753A4BBDF74F75E
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:03,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:03,781 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,783 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:03,783 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 60DCD32114994EF7B753A4BBDF74F75E initialized RecordReader from event
2016-05-18 00:39:03,784 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:03,785 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 60DCD32114994EF7B753A4BBDF74F75E
2016-05-18 00:39:03,785 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 5B74863A117748FEA0713DE844710519
2016-05-18 00:39:03,785 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 5B74863A117748FEA0713DE844710519
2016-05-18 00:39:03,787 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - E6BFBFADE4F2422B812F99A77DDFAAAA using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@3bd36d4c, comparator=cascading.tuple.hadoop.util.TupleComparator@35acb743, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:03,788 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for E6BFBFADE4F2422B812F99A77DDFAAAA: , UsingHashComparator=false
2016-05-18 00:39:03,800 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:03,800 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:03,800 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for E6BFBFADE4F2422B812F99A77DDFAAAA: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:03,800 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - E6BFBFADE4F2422B812F99A77DDFAAAA: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:03,801 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:03,801 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C92F0EFF4C7C412E905C988861769BB0, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:03,808 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,808 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C92F0EFF4C7C412E905C988861769BB0, mem on close (mb), free: 116, total: 327, max: 672
2016-05-18 00:39:03,808 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - E6BFBFADE4F2422B812F99A77DDFAAAA: Starting flush of map output
2016-05-18 00:39:03,809 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - E6BFBFADE4F2422B812F99A77DDFAAAA: Span0.length = 37, perItem = 17
2016-05-18 00:39:03,810 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - E6BFBFADE4F2422B812F99A77DDFAAAA: done sorting span=0, length=37, time=1
2016-05-18 00:39:03,810 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - E6BFBFADE4F2422B812F99A77DDFAAAA: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:03,811 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - E6BFBFADE4F2422B812F99A77DDFAAAA: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-local-dir/output/attempt_1463531943541_0001_1_00_000000_0_10029_0/file.out
2016-05-18 00:39:03,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:03 Completed running task attempt: attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,816 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:03,816 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531943541_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:03,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531943541_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531943802, Process_Duration=6, Process_End_Time=1463531943808, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:03,818 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531943541_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,818 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, taskAttemptId=attempt_1463531943541_0001_1_00_000000_0, creationTime=1463531943751, allocationTime=1463531943757, startTime=1463531943760, finishTime=1463531943818, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:03,818 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:03,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,820 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531943541_0001_00_000001
2016-05-18 00:39:03,820 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531943541_0001_1_00_000000_0
2016-05-18 00:39:03,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_FINISHED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, taskId=task_1463531943541_0001_1_00_000000, startTime=1463531943760, finishTime=1463531943820, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463531943541_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531943802, Process_Duration=6, Process_End_Time=1463531943808, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:03,821 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:03,821 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_FINISHED]: vertexName=C92F0EFF4C7C412E905C988861769BB0, vertexId=vertex_1463531943541_0001_1_00, initRequestedTime=1463531943712, initedTime=1463531943739, startRequestedTime=1463531943717, startedTime=1463531943740, finishTime=1463531943821, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531943802, Process_Duration=6, Process_End_Time=1463531943808, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531943760, firstTasksToStart=[ task_1463531943541_0001_1_00_000000 ], lastTaskFinishTime=1463531943818, lastTasksToFinish=[ task_1463531943541_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531943541_0001_1_00_000000 ], longestDurationTasks=[ task_1463531943541_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:03,822 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:03,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531943541_0001_00_000001, stoppedTime=1463531943823, exitStatus=0
2016-05-18 00:39:03,823 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] attempt: attempt_1463531943541_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:03,823 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:03,823 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E6BFBFADE4F2422B812F99A77DDFAAAA
2016-05-18 00:39:03,823 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,823 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=C92F0EFF4C7C412E905C988861769BB0, destinationVertexName=E6BFBFADE4F2422B812F99A77DDFAAAA. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:03,824 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,824 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531943541_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:03,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_STARTED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, taskId=task_1463531943541_0001_1_01_000000, scheduledTime=1463531943824, launchTime=1463531943824
2016-05-18 00:39:03,824 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531943541_0001_00_000002, launchTime=1463531943826
2016-05-18 00:39:03,827 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531943541_0001_00_000002
2016-05-18 00:39:03,828 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531943541_0001_1_01_000000_0] started. Is using containerId: [container_1463531943541_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:03,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0, startTime=1463531943828, containerId=container_1463531943541_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:03,829 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531943541_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03 Starting to run new task attempt: attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,830 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,830 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:03,830 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531943541_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:03,830 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531943541_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:03,831 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531943541_0001_00_000001, Exiting
2016-05-18 00:39:03,831 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531943541_0001_00_000001
2016-05-18 00:39:03,831 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531943541_0001_00_000001
2016-05-18 00:39:03,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: E6BFBFADE4F2422B812F99A77DDFAAAA, VertexParallelism: 1, TaskAttemptID:attempt_1463531943541_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C92F0EFF4C7C412E905C988861769BB0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CC847EB21BA546CD84F8A2B3BE978D48, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:03,832 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531943541_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:03,832 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531943541_0001_00_000001 completed successfully
2016-05-18 00:39:03,832 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:03,832 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,834 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531943541_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,835 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531943541_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:03,836 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:03,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:03,845 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E6BFBFADE4F2422B812F99A77DDFAAAA, ordinal: 1
2016-05-18 00:39:03,846 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6BFBFADE4F2422B812F99A77DDFAAAA, mem on start (mb), free: 111, total: 327, max: 672
2016-05-18 00:39:03,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:03,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:03,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:03,846 INFO output.MROutput (MROutput.java:initialize(421)) - CC847EB21BA546CD84F8A2B3BE978D48: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:03,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:03,846 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C92F0EFF4C7C412E905C988861769BB0:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [CC847EB21BA546CD84F8A2B3BE978D48:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:03,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:03,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:03,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:03,847 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C92F0EFF4C7C412E905C988861769BB0:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [CC847EB21BA546CD84F8A2B3BE978D48:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:03,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:03,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C92F0EFF4C7C412E905C988861769BB0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:03,848 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - C92F0EFF4C7C412E905C988861769BB0: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:03,848 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - C92F0EFF4C7C412E905C988861769BB0: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:03,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:03,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:03,849 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: C92F0EFF4C7C412E905C988861769BB0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:03,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C92F0EFF4C7C412E905C988861769BB0
2016-05-18 00:39:03,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:03,851 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,855 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:03,855 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C92F0EFF4C7C412E905C988861769BB0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:03,855 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 5B74863A117748FEA0713DE844710519
2016-05-18 00:39:03,855 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"], id: CC847EB21BA546CD84F8A2B3BE978D48
2016-05-18 00:39:03,856 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-05-18 00:39:03,856 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 5B74863A117748FEA0713DE844710519, for 1 inputs
2016-05-18 00:39:03,857 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531943541_0001_1_00_000000_0_10029} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531943857, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:03,857 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : C92F0EFF4C7C412E905C988861769BB0
2016-05-18 00:39:03,857 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:03,858 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: C92F0EFF4C7C412E905C988861769BB0, wasInterrupted=false
2016-05-18 00:39:03,858 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:03,858 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - C92F0EFF4C7C412E905C988861769BB0: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:03,858 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:03,859 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001_wd/localmode-local-dir/output/attempt_1463531943541_0001_1_00_000000_0_10029/file.out, len=593, isLocal=true
2016-05-18 00:39:03,860 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:03,860 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:03,860 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:03,860 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:03,860 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : C92F0EFF4C7C412E905C988861769BB0
2016-05-18 00:39:03,860 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - C92F0EFF4C7C412E905C988861769BB0: Shuffle Runner thread complete
2016-05-18 00:39:03,860 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E6BFBFADE4F2422B812F99A77DDFAAAA, all 1 inputs ready in: 00:00:00.004
2016-05-18 00:39:03,861 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - C92F0EFF4C7C412E905C988861769BB0: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:03,867 INFO output.MROutput (MROutput.java:close(526)) - CC847EB21BA546CD84F8A2B3BE978D48 closed
2016-05-18 00:39:03,868 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531943541_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:03,868 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,869 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6BFBFADE4F2422B812F99A77DDFAAAA, mem on close (mb), free: 107, total: 327, max: 672
2016-05-18 00:39:03,869 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: C92F0EFF4C7C412E905C988861769BB0
2016-05-18 00:39:03,869 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C92F0EFF4C7C412E905C988861769BB0: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:03,869 INFO output.MROutput (MROutput.java:close(526)) - CC847EB21BA546CD84F8A2B3BE978D48 closed
2016-05-18 00:39:03,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:03 Completed running task attempt: attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,870 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:03,870 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531943541_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:03,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531943541_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531943860, Process_Duration=8, Process_End_Time=1463531943868, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:03,872 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531943541_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, taskAttemptId=attempt_1463531943541_0001_1_01_000000_0, creationTime=1463531943824, allocationTime=1463531943825, startTime=1463531943828, finishTime=1463531943872, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:03,872 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:03,873 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:03,873 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531943541_0001_1_01_000000_0
2016-05-18 00:39:03,873 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,873 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531943541_0001_00_000002
2016-05-18 00:39:03,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:TASK_FINISHED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, taskId=task_1463531943541_0001_1_01_000000, startTime=1463531943828, finishTime=1463531943873, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463531943541_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531943860, Process_Duration=8, Process_End_Time=1463531943868, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:03,874 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:03,874 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA]
2016-05-18 00:39:03,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:VERTEX_FINISHED]: vertexName=E6BFBFADE4F2422B812F99A77DDFAAAA, vertexId=vertex_1463531943541_0001_1_01, initRequestedTime=1463531943717, initedTime=1463531943724, startRequestedTime=1463531943740, startedTime=1463531943741, finishTime=1463531943874, timeTaken=133, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531943860, Process_Duration=8, Process_End_Time=1463531943868, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531943828, firstTasksToStart=[ task_1463531943541_0001_1_01_000000 ], lastTaskFinishTime=1463531943872, lastTasksToFinish=[ task_1463531943541_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531943541_0001_1_01_000000 ], longestDurationTasks=[ task_1463531943541_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:03,875 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:03,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531943541_0001_00_000002, stoppedTime=1463531943876, exitStatus=0
2016-05-18 00:39:03,876 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:03,876 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,876 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531943541_0001_1_00 [C92F0EFF4C7C412E905C988861769BB0]
2016-05-18 00:39:03,876 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531943541_0001_1
2016-05-18 00:39:03,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531943541_0001_1
2016-05-18 00:39:03,877 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: CC847EB21BA546CD84F8A2B3BE978D48 for vertex: vertex_1463531943541_0001_1_01 [E6BFBFADE4F2422B812F99A77DDFAAAA], outputName: CC847EB21BA546CD84F8A2B3BE978D48
2016-05-18 00:39:03,877 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531943541_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:03,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531943541_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:03,883 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531943541_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:03,884 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531943541_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:03,884 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531943541_0001_00_000002, Exiting
2016-05-18 00:39:03,884 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531943541_0001_00_000002
2016-05-18 00:39:03,884 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531943541_0001_00_000002
2016-05-18 00:39:03,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531943541_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:03,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531943541_0001_00_000002 completed successfully
2016-05-18 00:39:03,885 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:CC847EB21BA546CD84F8A2B3BE978D48 of vertex/vertexGroup:E6BFBFADE4F2422B812F99A77DDFAAAA isVertexGroupOutput:false
2016-05-18 00:39:03,885 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:03,888 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531943541_0001_1, queueSize=0
2016-05-18 00:39:03,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531943541_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531943541_0001_1, startTime=1463531943712, finishTime=1463531943885, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=360, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063887662, Process_Duration=14, Process_End_Time=2927063887676, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:03,890 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531943541_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:03,890 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531943541_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:03 Completed Dag: dag_1463531943541_0001_1
2016-05-18 00:39:03,890 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531943541_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:03,890 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531943541_0001_1_post
2016-05-18 00:39:03,890 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:03,890 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:03,891 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:03,891 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false, with id=dag_1463531943541_0001_1
2016-05-18 00:39:03,891 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9] comprehensivesort/(1/1) ...ive/true_null_false_false, with id=dag_1463531943541_0001_1
2016-05-18 00:39:03,899 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1463531943541_0001
2016-05-18 00:39:03,902 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:03,902 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:03,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:03,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:03,902 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:03,902 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:03,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:03,903 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:03,904 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:03,904 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:03,904 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:03,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:03,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:03,906 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:03,910 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/CA71EA203CAF4CD2B5A10149F3043D4F/C445A73695B741E6893A56BB93F530F9/.tez/application_1463531943541_0001
2016-05-18 00:39:03,910 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false/_temporary
2016-05-18 00:39:03,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.390
2016-05-18 00:39:03,999 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:04,000 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:04,059 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 00:39:04,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-18 00:39:04,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:04,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:04,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:04,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:04,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:04,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:04,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_true_false
2016-05-18 00:39:04,196 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:04,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:04,214 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:04,215 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:04,215 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:04,225 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001 doesn't exist and is created
2016-05-18 00:39:04,243 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:04,243 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:04,245 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd
2016-05-18 00:39:04,255 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531944215_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:04,256 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:04,256 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:04,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:04,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:04,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:04,258 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:04,259 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:04,260 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:04,260 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:04,261 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:04,261 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:04,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:04,262 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:04,262 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:04,264 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:04,265 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:04,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:04,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-log-dir
2016-05-18 00:39:04,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-log-dir/history.txt.appattempt_1463531944215_0001_000000, maxErrors=10
2016-05-18 00:39:04,266 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:04,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531944215_0001_000000, appSubmitTime=1463531944254, launchTime=1463531944255
2016-05-18 00:39:04,274 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53764
2016-05-18 00:39:04,274 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531944215
2016-05-18 00:39:04,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:04,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531944215_0001_000000, startTime=1463531944277
2016-05-18 00:39:04,278 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:04,343 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:04,343 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:04,343 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1463531944215_0001, dagName=[3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-05-18 00:39:04,363 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-05-18 00:39:04,363 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:04,365 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531944215_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-log-dir/dag_1463531944215_0001_1.dot
2016-05-18 00:39:04,366 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-log-dir/dag_1463531944215_0001_1-tez-dag.pb.txt
2016-05-18 00:39:04,376 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531944215_0001_1
2016-05-18 00:39:04,376 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,376 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-05-18 00:39:04 Running Dag: dag_1463531944215_0001_1
2016-05-18 00:39:04,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531944215_0001_1, submitTime=1463531944363
2016-05-18 00:39:04,388 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001/tez-conf.pb" } size: 36903 timestamp: 1463531944000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:04,388 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,388 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:04,390 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001/tez-conf.pb" } size: 36903 timestamp: 1463531944000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:04,390 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,390 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:04,391 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:04,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531944215_0001_1, initTime=1463531944392
2016-05-18 00:39:04,394 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944215_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:04,394 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:04,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:DAG_STARTED]: dagID=dag_1463531944215_0001_1, startTime=1463531944394
2016-05-18 00:39:04,394 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944215_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:04,394 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B3362BB638B849998558B18CCFB35FB5 : {B649D34FCCA2456C9D572012F01D3974={InputName=B649D34FCCA2456C9D572012F01D3974}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:04,395 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B649D34FCCA2456C9D572012F01D3974, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:04,395 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,395 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,395 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,395 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,396 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:04,396 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,396 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1463531944215_0001, dagName=[3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-05-18 00:39:04,398 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B649D34FCCA2456C9D572012F01D3974 on vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,399 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:04,399 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,400 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531944215_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:04,403 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7], numAdditionalOutputs=1
2016-05-18 00:39:04,403 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=55D3B1E351744205B71F4A176E3F0A71, vertex=vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:04,403 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B649D34FCCA2456C9D572012F01D3974 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:04,406 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F0D14C50FBBA4ECB83A97B264FBA6AF7:55D3B1E351744205B71F4A176E3F0A71 using oldmapred API
2016-05-18 00:39:04,406 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=55D3B1E351744205B71F4A176E3F0A71, vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:04,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, vertexId=vertex_1463531944215_0001_1_01, initRequestedTime=1463531944396, initedTime=1463531944403, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:04,408 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:04,408 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:04,409 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:04,409 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:04,409 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B649D34FCCA2456C9D572012F01D3974 on vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,408 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:04,412 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:04,412 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B649D34FCCA2456C9D572012F01D3974=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:04,414 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] parallelism set to 1
2016-05-18 00:39:04,414 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:04,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3362BB638B849998558B18CCFB35FB5, vertexId=vertex_1463531944215_0001_1_00, initRequestedTime=1463531944394, initedTime=1463531944415, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:04,415 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] with distanceFromRoot: 0
2016-05-18 00:39:04,415 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:04,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531944215_0001_1_00, startRequestedTime=1463531944396, startedTime=1463531944415
2016-05-18 00:39:04,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531944215_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:04,416 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:04,416 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531944215_0001_1_00 for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] numStartedSources: 1 numSources: 1
2016-05-18 00:39:04,416 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] with distanceFromRoot: 1
2016-05-18 00:39:04,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531944215_0001_1_01, startRequestedTime=1463531944416, startedTime=1463531944416
2016-05-18 00:39:04,421 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:04,425 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F0D14C50FBBA4ECB83A97B264FBA6AF7 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:04,426 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B3362BB638B849998558B18CCFB35FB5 in vertex: F0D14C50FBBA4ECB83A97B264FBA6AF7 numBipartiteSourceTasks: 1
2016-05-18 00:39:04,429 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B3362BB638B849998558B18CCFB35FB5
2016-05-18 00:39:04,430 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_STARTED]: vertexName=B3362BB638B849998558B18CCFB35FB5, taskId=task_1463531944215_0001_1_00_000000, scheduledTime=1463531944430, launchTime=1463531944430
2016-05-18 00:39:04,431 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,431 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531944215_0001, with dag id: dag_1463531944215_0001_1
2016-05-18 00:39:04,433 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:04,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531944215_0001_00_000001, launchTime=1463531944438
2016-05-18 00:39:04,440 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944215_0001_00_000001
2016-05-18 00:39:04,442 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944215_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,442 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531944215_0001_1_00_000000_0] started. Is using containerId: [container_1463531944215_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:04 Starting to run new task attempt: attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,442 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3362BB638B849998558B18CCFB35FB5, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0, startTime=1463531944442, containerId=container_1463531944215_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:04,443 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,443 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:04,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: B3362BB638B849998558B18CCFB35FB5, VertexParallelism: 1, TaskAttemptID:attempt_1463531944215_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B649D34FCCA2456C9D572012F01D3974, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:04,444 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:04,445 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,449 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531944215_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:04,449 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531944215_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:04,453 INFO input.MRInput (MRInput.java:initialize(440)) - B649D34FCCA2456C9D572012F01D3974 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:04,453 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B649D34FCCA2456C9D572012F01D3974
2016-05-18 00:39:04,455 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:04,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3362BB638B849998558B18CCFB35FB5, ordinal: 0
2016-05-18 00:39:04,464 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3362BB638B849998558B18CCFB35FB5, mem on start (mb), free: 223, total: 339, max: 672
2016-05-18 00:39:04,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:04,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:04,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:04,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:04,464 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B649D34FCCA2456C9D572012F01D3974:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F0D14C50FBBA4ECB83A97B264FBA6AF7:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:04,465 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:04,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:04,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:04,467 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B649D34FCCA2456C9D572012F01D3974:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F0D14C50FBBA4ECB83A97B264FBA6AF7:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B649D34FCCA2456C9D572012F01D3974 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B649D34FCCA2456C9D572012F01D3974
2016-05-18 00:39:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:04,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:04,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:04,469 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,472 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:04,473 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:04,473 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: B649D34FCCA2456C9D572012F01D3974
2016-05-18 00:39:04,473 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 1FEBAC323F7643A897E23B0ED80823DC
2016-05-18 00:39:04,473 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B649D34FCCA2456C9D572012F01D3974 initialized RecordReader from event
2016-05-18 00:39:04,473 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1FEBAC323F7643A897E23B0ED80823DC
2016-05-18 00:39:04,475 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F0D14C50FBBA4ECB83A97B264FBA6AF7 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@7ccf26f3, comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@4065b89, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:04,475 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F0D14C50FBBA4ECB83A97B264FBA6AF7: , UsingHashComparator=false
2016-05-18 00:39:04,488 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:04,488 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:04,488 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F0D14C50FBBA4ECB83A97B264FBA6AF7: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:04,488 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:04,489 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:04,489 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3362BB638B849998558B18CCFB35FB5, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:04,495 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,495 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3362BB638B849998558B18CCFB35FB5, mem on close (mb), free: 120, total: 339, max: 672
2016-05-18 00:39:04,495 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: Starting flush of map output
2016-05-18 00:39:04,496 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: Span0.length = 37, perItem = 17
2016-05-18 00:39:04,496 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: done sorting span=0, length=37, time=0
2016-05-18 00:39:04,497 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:04,498 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F0D14C50FBBA4ECB83A97B264FBA6AF7: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-local-dir/output/attempt_1463531944215_0001_1_00_000000_0_10035_0/file.out
2016-05-18 00:39:04,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:04 Completed running task attempt: attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,502 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:04,503 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531944215_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:04,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531944215_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531944489, Process_Duration=6, Process_End_Time=1463531944495, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:04,504 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531944215_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:04,505 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:04,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3362BB638B849998558B18CCFB35FB5, taskAttemptId=attempt_1463531944215_0001_1_00_000000_0, creationTime=1463531944430, allocationTime=1463531944437, startTime=1463531944442, finishTime=1463531944505, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:04,505 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:04,505 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,505 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531944215_0001_1_00_000000_0
2016-05-18 00:39:04,505 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944215_0001_00_000001
2016-05-18 00:39:04,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_FINISHED]: vertexName=B3362BB638B849998558B18CCFB35FB5, taskId=task_1463531944215_0001_1_00_000000, startTime=1463531944442, finishTime=1463531944505, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463531944215_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531944489, Process_Duration=6, Process_End_Time=1463531944495, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:04,507 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:04,507 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_FINISHED]: vertexName=B3362BB638B849998558B18CCFB35FB5, vertexId=vertex_1463531944215_0001_1_00, initRequestedTime=1463531944394, initedTime=1463531944415, startRequestedTime=1463531944396, startedTime=1463531944415, finishTime=1463531944507, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531944489, Process_Duration=6, Process_End_Time=1463531944495, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531944442, firstTasksToStart=[ task_1463531944215_0001_1_00_000000 ], lastTaskFinishTime=1463531944505, lastTasksToFinish=[ task_1463531944215_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531944215_0001_1_00_000000 ], longestDurationTasks=[ task_1463531944215_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:04,509 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:04,510 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] attempt: attempt_1463531944215_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:04,510 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F0D14C50FBBA4ECB83A97B264FBA6AF7
2016-05-18 00:39:04,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531944215_0001_00_000001, stoppedTime=1463531944510, exitStatus=0
2016-05-18 00:39:04,510 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B3362BB638B849998558B18CCFB35FB5, destinationVertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:04,511 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,511 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:04,511 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:04,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531944215_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:04,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_STARTED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, taskId=task_1463531944215_0001_1_01_000000, scheduledTime=1463531944511, launchTime=1463531944511
2016-05-18 00:39:04,512 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531944215_0001_00_000002, launchTime=1463531944514
2016-05-18 00:39:04,515 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944215_0001_00_000002
2016-05-18 00:39:04,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531944215_0001_1_01_000000_0] started. Is using containerId: [container_1463531944215_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:04,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0, startTime=1463531944515, containerId=container_1463531944215_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:04,516 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944215_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,517 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531944215_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:04,517 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944215_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:04,517 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531944215_0001_00_000001, Exiting
2016-05-18 00:39:04,517 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531944215_0001_00_000001
2016-05-18 00:39:04,517 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531944215_0001_00_000001
2016-05-18 00:39:04 Starting to run new task attempt: attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,518 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531944215_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:04,519 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531944215_0001_00_000001 completed successfully
2016-05-18 00:39:04,519 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:04,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: F0D14C50FBBA4ECB83A97B264FBA6AF7, VertexParallelism: 1, TaskAttemptID:attempt_1463531944215_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B3362BB638B849998558B18CCFB35FB5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=55D3B1E351744205B71F4A176E3F0A71, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:04,520 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:04,520 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,523 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531944215_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,523 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531944215_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:04,525 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:04,528 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:04,533 INFO output.MROutput (MROutput.java:initialize(421)) - 55D3B1E351744205B71F4A176E3F0A71: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:04,535 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F0D14C50FBBA4ECB83A97B264FBA6AF7, ordinal: 1
2016-05-18 00:39:04,536 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0D14C50FBBA4ECB83A97B264FBA6AF7, mem on start (mb), free: 115, total: 339, max: 672
2016-05-18 00:39:04,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:04,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:04,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:04,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:04,536 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[55D3B1E351744205B71F4A176E3F0A71:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B3362BB638B849998558B18CCFB35FB5:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:04,536 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:04,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:04,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:04,537 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[55D3B1E351744205B71F4A176E3F0A71:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B3362BB638B849998558B18CCFB35FB5:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:04,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:04,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B3362BB638B849998558B18CCFB35FB5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:04,538 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B3362BB638B849998558B18CCFB35FB5: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:04,538 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B3362BB638B849998558B18CCFB35FB5: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:04,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:04,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:04,540 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B3362BB638B849998558B18CCFB35FB5 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:04,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B3362BB638B849998558B18CCFB35FB5
2016-05-18 00:39:04,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:04,544 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,544 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B3362BB638B849998558B18CCFB35FB5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:04,546 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531944215_0001_1_00_000000_0_10035} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531944546, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:04,546 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B3362BB638B849998558B18CCFB35FB5
2016-05-18 00:39:04,546 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:04,547 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B3362BB638B849998558B18CCFB35FB5, wasInterrupted=false
2016-05-18 00:39:04,547 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:04,550 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:04,550 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:04,551 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 1FEBAC323F7643A897E23B0ED80823DC
2016-05-18 00:39:04,552 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"], id: 55D3B1E351744205B71F4A176E3F0A71
2016-05-18 00:39:04,551 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001_wd/localmode-local-dir/output/attempt_1463531944215_0001_1_00_000000_0_10035/file.out, len=593, isLocal=true
2016-05-18 00:39:04,552 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:04,552 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:04,553 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:04,553 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:04,553 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B3362BB638B849998558B18CCFB35FB5
2016-05-18 00:39:04,553 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B3362BB638B849998558B18CCFB35FB5: Shuffle Runner thread complete
2016-05-18 00:39:04,554 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-05-18 00:39:04,554 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1FEBAC323F7643A897E23B0ED80823DC, for 1 inputs
2016-05-18 00:39:04,555 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F0D14C50FBBA4ECB83A97B264FBA6AF7, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:04,556 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B3362BB638B849998558B18CCFB35FB5: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseTupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:04,561 INFO output.MROutput (MROutput.java:close(526)) - 55D3B1E351744205B71F4A176E3F0A71 closed
2016-05-18 00:39:04,562 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531944215_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:04,572 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,573 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0D14C50FBBA4ECB83A97B264FBA6AF7, mem on close (mb), free: 314, total: 337, max: 672
2016-05-18 00:39:04,573 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B3362BB638B849998558B18CCFB35FB5
2016-05-18 00:39:04,574 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B3362BB638B849998558B18CCFB35FB5: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:04,575 INFO output.MROutput (MROutput.java:close(526)) - 55D3B1E351744205B71F4A176E3F0A71 closed
2016-05-18 00:39:04,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:04 Completed running task attempt: attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,575 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:04,575 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531944215_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:04,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531944215_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531944555, Process_Duration=17, Process_End_Time=1463531944572, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:04,578 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531944215_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:04,578 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:04,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, taskAttemptId=attempt_1463531944215_0001_1_01_000000_0, creationTime=1463531944512, allocationTime=1463531944513, startTime=1463531944515, finishTime=1463531944578, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:04,579 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531944215_0001_1_01_000000_0
2016-05-18 00:39:04,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:04,580 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:TASK_FINISHED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, taskId=task_1463531944215_0001_1_01_000000, startTime=1463531944515, finishTime=1463531944579, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463531944215_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531944555, Process_Duration=17, Process_End_Time=1463531944572, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:04,580 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:04,580 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7]
2016-05-18 00:39:04,580 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944215_0001_00_000002
2016-05-18 00:39:04,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:VERTEX_FINISHED]: vertexName=F0D14C50FBBA4ECB83A97B264FBA6AF7, vertexId=vertex_1463531944215_0001_1_01, initRequestedTime=1463531944396, initedTime=1463531944403, startRequestedTime=1463531944416, startedTime=1463531944416, finishTime=1463531944580, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531944555, Process_Duration=17, Process_End_Time=1463531944572, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531944515, firstTasksToStart=[ task_1463531944215_0001_1_01_000000 ], lastTaskFinishTime=1463531944578, lastTasksToFinish=[ task_1463531944215_0001_1_01_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531944215_0001_1_01_000000 ], longestDurationTasks=[ task_1463531944215_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:04,582 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:04,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531944215_0001_00_000002, stoppedTime=1463531944582, exitStatus=0
2016-05-18 00:39:04,583 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:04,583 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:04,583 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531944215_0001_1_00 [B3362BB638B849998558B18CCFB35FB5]
2016-05-18 00:39:04,583 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531944215_0001_1
2016-05-18 00:39:04,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531944215_0001_1
2016-05-18 00:39:04,584 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 55D3B1E351744205B71F4A176E3F0A71 for vertex: vertex_1463531944215_0001_1_01 [F0D14C50FBBA4ECB83A97B264FBA6AF7], outputName: 55D3B1E351744205B71F4A176E3F0A71
2016-05-18 00:39:04,584 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944215_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:04,586 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531944215_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:04,592 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:55D3B1E351744205B71F4A176E3F0A71 of vertex/vertexGroup:F0D14C50FBBA4ECB83A97B264FBA6AF7 isVertexGroupOutput:false
2016-05-18 00:39:04,592 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:04,592 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531944215_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:04,592 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944215_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:04,593 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531944215_0001_00_000002, Exiting
2016-05-18 00:39:04,593 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531944215_0001_00_000002
2016-05-18 00:39:04,593 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531944215_0001_00_000002
2016-05-18 00:39:04,593 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531944215_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:04,593 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531944215_0001_00_000002 completed successfully
2016-05-18 00:39:04,595 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531944215_0001_1, queueSize=0
2016-05-18 00:39:04,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944215_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531944215_0001_1, startTime=1463531944394, finishTime=1463531944592, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=380, AM_GC_TIME_MILLIS=9, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063889044, Process_Duration=23, Process_End_Time=2927063889067, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=4, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:04,597 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531944215_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:04,597 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944215_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:04 Completed Dag: dag_1463531944215_0001_1
2016-05-18 00:39:04,598 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531944215_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:04,599 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531944215_0001_1_post
2016-05-18 00:39:04,599 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:04,600 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:04,600 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:04,601 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false, with id=dag_1463531944215_0001_1
2016-05-18 00:39:04,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD] comprehensivesort/(1/1) ...ive/false_null_true_false, with id=dag_1463531944215_0001_1
2016-05-18 00:39:04,605 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1463531944215_0001
2016-05-18 00:39:04,608 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:04,608 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:04,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:04,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:04,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:04,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:04,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:04,610 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:04,610 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:04,610 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:04,611 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:04,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:04,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:04,613 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:04,617 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/3908EAA5475C40C4AD7B0549658CDAD2/BF172F5B8B724B049C7AC1E6ECC1E8BD/.tez/application_1463531944215_0001
2016-05-18 00:39:04,617 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false/_temporary
2016-05-18 00:39:04,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.422
2016-05-18 00:39:04,693 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:04,693 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:04,745 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 00:39:04,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-18 00:39:04,748 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:04,748 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:04,749 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:04,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:04,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false
2016-05-18 00:39:04,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:04,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:04,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-05-18 00:39:04,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:04,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:04,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:04,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_true_false
2016-05-18 00:39:04,839 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:04,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:04,858 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:04,859 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:04,859 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:04,867 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001 doesn't exist and is created
2016-05-18 00:39:04,881 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:04,881 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:04,882 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd
2016-05-18 00:39:04,892 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531944858_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:04,893 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:04,893 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:04,894 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:04,894 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:04,894 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:04,895 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:04,896 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:04,897 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:04,897 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:04,897 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:04,898 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:04,898 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:04,898 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:04,898 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:04,898 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:04,898 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:04,899 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:04,899 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:04,899 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:04,901 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:04,902 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:04,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:04,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-log-dir
2016-05-18 00:39:04,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-log-dir/history.txt.appattempt_1463531944858_0001_000000, maxErrors=10
2016-05-18 00:39:04,903 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:04,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531944858_0001_000000, appSubmitTime=1463531944892, launchTime=1463531944892
2016-05-18 00:39:04,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:04,914 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35841
2016-05-18 00:39:04,915 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531944858
2016-05-18 00:39:04,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531944858_0001_000000, startTime=1463531944916
2016-05-18 00:39:04,917 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:04,981 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:04,982 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:04,982 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1463531944858_0001, dagName=[E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-05-18 00:39:05,000 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-05-18 00:39:05,001 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:05,002 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531944858_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-log-dir/dag_1463531944858_0001_1.dot
2016-05-18 00:39:05,002 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-log-dir/dag_1463531944858_0001_1-tez-dag.pb.txt
2016-05-18 00:39:05,009 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531944858_0001_1
2016-05-18 00:39:05,009 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,009 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-05-18 00:39:05 Running Dag: dag_1463531944858_0001_1
2016-05-18 00:39:05,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531944858_0001_1, submitTime=1463531945001
2016-05-18 00:39:05,020 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001/tez-conf.pb" } size: 36903 timestamp: 1463531944000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:05,020 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,021 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:05,022 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001/tez-conf.pb" } size: 36903 timestamp: 1463531944000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:05,022 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,022 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:05,023 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:05,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531944858_0001_1, initTime=1463531945023
2016-05-18 00:39:05,024 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944858_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:05,025 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:05,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:DAG_STARTED]: dagID=dag_1463531944858_0001_1, startTime=1463531945025
2016-05-18 00:39:05,025 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944858_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:05,025 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D9E094AD9ADE41B29411290A653CC59B : {0653E16112E64A0CA829C76731EAC835={InputName=0653E16112E64A0CA829C76731EAC835}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:05,026 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 0653E16112E64A0CA829C76731EAC835, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:05,026 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,026 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,026 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,026 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,027 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1463531944858_0001, dagName=[E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-05-18 00:39:05,027 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:05,027 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,027 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 0653E16112E64A0CA829C76731EAC835 on vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,030 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 0653E16112E64A0CA829C76731EAC835 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:05,031 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:05,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531944858_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:05,034 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50], numAdditionalOutputs=1
2016-05-18 00:39:05,035 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=7BD3DB37F2074D05AB1F2A8425D78D70, vertex=vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:05,035 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:05,035 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:05,036 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:05,036 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:05,037 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 0653E16112E64A0CA829C76731EAC835 on vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,038 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DB1FE45299D04134B62B3CD7C4AEBE50:7BD3DB37F2074D05AB1F2A8425D78D70 using oldmapred API
2016-05-18 00:39:05,038 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=7BD3DB37F2074D05AB1F2A8425D78D70, vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:05,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, vertexId=vertex_1463531944858_0001_1_01, initRequestedTime=1463531945027, initedTime=1463531945034, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:05,040 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:05,041 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:05,041 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {0653E16112E64A0CA829C76731EAC835=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:05,042 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] parallelism set to 1
2016-05-18 00:39:05,042 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:05,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, vertexId=vertex_1463531944858_0001_1_00, initRequestedTime=1463531945025, initedTime=1463531945043, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:05,043 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] with distanceFromRoot: 0
2016-05-18 00:39:05,043 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:05,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531944858_0001_1_00, startRequestedTime=1463531945027, startedTime=1463531945043
2016-05-18 00:39:05,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531944858_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:05,044 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:05,044 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531944858_0001_1_00 for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] numStartedSources: 1 numSources: 1
2016-05-18 00:39:05,044 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] with distanceFromRoot: 1
2016-05-18 00:39:05,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531944858_0001_1_01, startRequestedTime=1463531945044, startedTime=1463531945044
2016-05-18 00:39:05,049 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:05,051 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D9E094AD9ADE41B29411290A653CC59B
2016-05-18 00:39:05,051 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_STARTED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, taskId=task_1463531944858_0001_1_00_000000, scheduledTime=1463531945052, launchTime=1463531945052
2016-05-18 00:39:05,052 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,058 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:05,058 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: DB1FE45299D04134B62B3CD7C4AEBE50 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:05,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531944858_0001_00_000001, launchTime=1463531945059
2016-05-18 00:39:05,060 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D9E094AD9ADE41B29411290A653CC59B in vertex: DB1FE45299D04134B62B3CD7C4AEBE50 numBipartiteSourceTasks: 1
2016-05-18 00:39:05,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531944858_0001, with dag id: dag_1463531944858_0001_1
2016-05-18 00:39:05,063 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944858_0001_00_000001
2016-05-18 00:39:05,065 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944858_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,066 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531944858_0001_1_00_000000_0] started. Is using containerId: [container_1463531944858_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:05,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0, startTime=1463531945065, containerId=container_1463531944858_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:05 Starting to run new task attempt: attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,066 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,066 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,066 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:05,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: D9E094AD9ADE41B29411290A653CC59B, VertexParallelism: 1, TaskAttemptID:attempt_1463531944858_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0653E16112E64A0CA829C76731EAC835, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB1FE45299D04134B62B3CD7C4AEBE50, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:05,067 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:05,068 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,068 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531944858_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:05,070 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531944858_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:05,073 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:05,076 INFO input.MRInput (MRInput.java:initialize(440)) - 0653E16112E64A0CA829C76731EAC835 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:05,076 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 0653E16112E64A0CA829C76731EAC835
2016-05-18 00:39:05,081 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9E094AD9ADE41B29411290A653CC59B, ordinal: 0
2016-05-18 00:39:05,081 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9E094AD9ADE41B29411290A653CC59B, mem on start (mb), free: 210, total: 337, max: 672
2016-05-18 00:39:05,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:05,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:05,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:05,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:05,082 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[0653E16112E64A0CA829C76731EAC835:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DB1FE45299D04134B62B3CD7C4AEBE50:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:05,082 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:05,082 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:05,082 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:05,083 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[0653E16112E64A0CA829C76731EAC835:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DB1FE45299D04134B62B3CD7C4AEBE50:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:05,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:05,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 0653E16112E64A0CA829C76731EAC835 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:05,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 0653E16112E64A0CA829C76731EAC835
2016-05-18 00:39:05,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:05,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:05,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:05,085 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,088 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:05,088 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 0653E16112E64A0CA829C76731EAC835 initialized RecordReader from event
2016-05-18 00:39:05,089 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:05,089 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 0653E16112E64A0CA829C76731EAC835
2016-05-18 00:39:05,090 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: D0FDCEAB620C4A5AB37B80C6D64F25BE
2016-05-18 00:39:05,090 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D0FDCEAB620C4A5AB37B80C6D64F25BE
2016-05-18 00:39:05,092 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - DB1FE45299D04134B62B3CD7C4AEBE50 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@584f3d72, comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@3a4b34a9, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:05,092 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for DB1FE45299D04134B62B3CD7C4AEBE50: , UsingHashComparator=false
2016-05-18 00:39:05,106 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:05,107 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:05,107 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for DB1FE45299D04134B62B3CD7C4AEBE50: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:05,107 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - DB1FE45299D04134B62B3CD7C4AEBE50: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:05,108 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:05,108 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9E094AD9ADE41B29411290A653CC59B, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:05,114 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,115 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9E094AD9ADE41B29411290A653CC59B, mem on close (mb), free: 108, total: 337, max: 672
2016-05-18 00:39:05,115 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - DB1FE45299D04134B62B3CD7C4AEBE50: Starting flush of map output
2016-05-18 00:39:05,115 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - DB1FE45299D04134B62B3CD7C4AEBE50: Span0.length = 37, perItem = 17
2016-05-18 00:39:05,116 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - DB1FE45299D04134B62B3CD7C4AEBE50: done sorting span=0, length=37, time=1
2016-05-18 00:39:05,116 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - DB1FE45299D04134B62B3CD7C4AEBE50: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:05,117 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - DB1FE45299D04134B62B3CD7C4AEBE50: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-local-dir/output/attempt_1463531944858_0001_1_00_000000_0_10040_0/file.out
2016-05-18 00:39:05,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:05 Completed running task attempt: attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,124 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:05,124 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531944858_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:05,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531944858_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531945108, Process_Duration=6, Process_End_Time=1463531945114, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:05,126 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531944858_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,128 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, taskAttemptId=attempt_1463531944858_0001_1_00_000000_0, creationTime=1463531945052, allocationTime=1463531945058, startTime=1463531945065, finishTime=1463531945126, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:05,128 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:05,129 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,129 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944858_0001_00_000001
2016-05-18 00:39:05,129 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531944858_0001_1_00_000000_0
2016-05-18 00:39:05,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_FINISHED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, taskId=task_1463531944858_0001_1_00_000000, startTime=1463531945065, finishTime=1463531945129, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463531944858_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531945108, Process_Duration=6, Process_End_Time=1463531945114, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:05,130 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:05,130 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_FINISHED]: vertexName=D9E094AD9ADE41B29411290A653CC59B, vertexId=vertex_1463531944858_0001_1_00, initRequestedTime=1463531945025, initedTime=1463531945043, startRequestedTime=1463531945027, startedTime=1463531945043, finishTime=1463531945130, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531945108, Process_Duration=6, Process_End_Time=1463531945114, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531945065, firstTasksToStart=[ task_1463531944858_0001_1_00_000000 ], lastTaskFinishTime=1463531945126, lastTasksToFinish=[ task_1463531944858_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531944858_0001_1_00_000000 ], longestDurationTasks=[ task_1463531944858_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:05,132 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:05,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531944858_0001_00_000001, stoppedTime=1463531945132, exitStatus=0
2016-05-18 00:39:05,132 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] attempt: attempt_1463531944858_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:05,132 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:05,132 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DB1FE45299D04134B62B3CD7C4AEBE50
2016-05-18 00:39:05,132 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D9E094AD9ADE41B29411290A653CC59B, destinationVertexName=DB1FE45299D04134B62B3CD7C4AEBE50. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:05,133 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,132 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,133 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531944858_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:05,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_STARTED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, taskId=task_1463531944858_0001_1_01_000000, scheduledTime=1463531945133, launchTime=1463531945133
2016-05-18 00:39:05,134 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531944858_0001_00_000002, launchTime=1463531945135
2016-05-18 00:39:05,136 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944858_0001_00_000002
2016-05-18 00:39:05,137 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531944858_0001_1_01_000000_0] started. Is using containerId: [container_1463531944858_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:05,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0, startTime=1463531945137, containerId=container_1463531944858_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:05,138 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944858_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05 Starting to run new task attempt: attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,138 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,139 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:05,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: DB1FE45299D04134B62B3CD7C4AEBE50, VertexParallelism: 1, TaskAttemptID:attempt_1463531944858_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D9E094AD9ADE41B29411290A653CC59B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7BD3DB37F2074D05AB1F2A8425D78D70, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:05,140 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:05,140 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531944858_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:05,140 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,141 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944858_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:05,141 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531944858_0001_00_000001, Exiting
2016-05-18 00:39:05,141 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531944858_0001_00_000001
2016-05-18 00:39:05,151 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531944858_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,151 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531944858_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:05,142 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531944858_0001_00_000001
2016-05-18 00:39:05,156 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:05,158 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531944858_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:05,159 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531944858_0001_00_000001 completed successfully
2016-05-18 00:39:05,162 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:05,165 INFO output.MROutput (MROutput.java:initialize(421)) - 7BD3DB37F2074D05AB1F2A8425D78D70: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:05,167 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DB1FE45299D04134B62B3CD7C4AEBE50, ordinal: 1
2016-05-18 00:39:05,167 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB1FE45299D04134B62B3CD7C4AEBE50, mem on start (mb), free: 309, total: 338, max: 672
2016-05-18 00:39:05,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:05,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:05,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:05,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:05,168 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7BD3DB37F2074D05AB1F2A8425D78D70:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [D9E094AD9ADE41B29411290A653CC59B:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:05,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:05,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:05,170 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:05,170 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7BD3DB37F2074D05AB1F2A8425D78D70:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [D9E094AD9ADE41B29411290A653CC59B:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:05,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:05,171 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D9E094AD9ADE41B29411290A653CC59B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:05,171 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D9E094AD9ADE41B29411290A653CC59B: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:05,172 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D9E094AD9ADE41B29411290A653CC59B: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:05,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:05,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:05,174 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D9E094AD9ADE41B29411290A653CC59B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:05,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D9E094AD9ADE41B29411290A653CC59B
2016-05-18 00:39:05,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:05,175 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,176 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D9E094AD9ADE41B29411290A653CC59B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:05,178 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531944858_0001_1_00_000000_0_10040} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531945178, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:05,179 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D9E094AD9ADE41B29411290A653CC59B
2016-05-18 00:39:05,179 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:05,179 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D9E094AD9ADE41B29411290A653CC59B, wasInterrupted=false
2016-05-18 00:39:05,180 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:05,180 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:05,181 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:05,181 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: D0FDCEAB620C4A5AB37B80C6D64F25BE
2016-05-18 00:39:05,182 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"], id: 7BD3DB37F2074D05AB1F2A8425D78D70
2016-05-18 00:39:05,182 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-05-18 00:39:05,182 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - D9E094AD9ADE41B29411290A653CC59B: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:05,183 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D0FDCEAB620C4A5AB37B80C6D64F25BE, for 1 inputs
2016-05-18 00:39:05,182 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001_wd/localmode-local-dir/output/attempt_1463531944858_0001_1_00_000000_0_10040/file.out, len=593, isLocal=true
2016-05-18 00:39:05,183 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:05,183 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:05,183 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:05,184 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:05,184 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D9E094AD9ADE41B29411290A653CC59B
2016-05-18 00:39:05,184 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DB1FE45299D04134B62B3CD7C4AEBE50, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:05,185 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D9E094AD9ADE41B29411290A653CC59B: Shuffle Runner thread complete
2016-05-18 00:39:05,186 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D9E094AD9ADE41B29411290A653CC59B: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseTupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:05,191 INFO output.MROutput (MROutput.java:close(526)) - 7BD3DB37F2074D05AB1F2A8425D78D70 closed
2016-05-18 00:39:05,191 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531944858_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:05,192 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,192 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB1FE45299D04134B62B3CD7C4AEBE50, mem on close (mb), free: 307, total: 338, max: 672
2016-05-18 00:39:05,193 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D9E094AD9ADE41B29411290A653CC59B
2016-05-18 00:39:05,193 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D9E094AD9ADE41B29411290A653CC59B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:05,194 INFO output.MROutput (MROutput.java:close(526)) - 7BD3DB37F2074D05AB1F2A8425D78D70 closed
2016-05-18 00:39:05,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:05 Completed running task attempt: attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,195 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:05,195 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531944858_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:05,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531944858_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531945185, Process_Duration=7, Process_End_Time=1463531945192, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:05,197 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531944858_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,197 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, taskAttemptId=attempt_1463531944858_0001_1_01_000000_0, creationTime=1463531945134, allocationTime=1463531945135, startTime=1463531945137, finishTime=1463531945197, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:05,197 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:05,198 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,198 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531944858_0001_1_01_000000_0
2016-05-18 00:39:05,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:TASK_FINISHED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, taskId=task_1463531944858_0001_1_01_000000, startTime=1463531945137, finishTime=1463531945198, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463531944858_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531945185, Process_Duration=7, Process_End_Time=1463531945192, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:05,199 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:05,199 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50]
2016-05-18 00:39:05,199 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531944858_0001_00_000002
2016-05-18 00:39:05,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:VERTEX_FINISHED]: vertexName=DB1FE45299D04134B62B3CD7C4AEBE50, vertexId=vertex_1463531944858_0001_1_01, initRequestedTime=1463531945027, initedTime=1463531945034, startRequestedTime=1463531945044, startedTime=1463531945044, finishTime=1463531945199, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531945185, Process_Duration=7, Process_End_Time=1463531945192, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531945137, firstTasksToStart=[ task_1463531944858_0001_1_01_000000 ], lastTaskFinishTime=1463531945197, lastTasksToFinish=[ task_1463531944858_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531944858_0001_1_01_000000 ], longestDurationTasks=[ task_1463531944858_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:05,201 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:05,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531944858_0001_00_000002, stoppedTime=1463531945201, exitStatus=0
2016-05-18 00:39:05,201 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:05,201 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,202 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531944858_0001_1_00 [D9E094AD9ADE41B29411290A653CC59B]
2016-05-18 00:39:05,202 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531944858_0001_1
2016-05-18 00:39:05,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531944858_0001_1
2016-05-18 00:39:05,202 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 7BD3DB37F2074D05AB1F2A8425D78D70 for vertex: vertex_1463531944858_0001_1_01 [DB1FE45299D04134B62B3CD7C4AEBE50], outputName: 7BD3DB37F2074D05AB1F2A8425D78D70
2016-05-18 00:39:05,204 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944858_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:05,204 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531944858_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:05,210 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:7BD3DB37F2074D05AB1F2A8425D78D70 of vertex/vertexGroup:DB1FE45299D04134B62B3CD7C4AEBE50 isVertexGroupOutput:false
2016-05-18 00:39:05,210 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,210 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531944858_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:05,210 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531944858_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:05,210 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531944858_0001_00_000002, Exiting
2016-05-18 00:39:05,210 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531944858_0001_00_000002
2016-05-18 00:39:05,210 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531944858_0001_00_000002
2016-05-18 00:39:05,211 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531944858_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:05,211 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531944858_0001_00_000002 completed successfully
2016-05-18 00:39:05,213 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531944858_0001_1, queueSize=0
2016-05-18 00:39:05,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531944858_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531944858_0001_1, startTime=1463531945025, finishTime=1463531945210, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=380, AM_GC_TIME_MILLIS=8, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063890293, Process_Duration=13, Process_End_Time=2927063890306, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:05,213 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531944858_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:05,214 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531944858_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:05 Completed Dag: dag_1463531944858_0001_1
2016-05-18 00:39:05,214 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531944858_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:05,214 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531944858_0001_1_post
2016-05-18 00:39:05,215 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:05,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:05,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false, with id=dag_1463531944858_0001_1
2016-05-18 00:39:05,215 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F] comprehensivesort/(1/1) ...sive/true_null_true_false, with id=dag_1463531944858_0001_1
2016-05-18 00:39:05,224 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1463531944858_0001
2016-05-18 00:39:05,228 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:05,228 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:05,228 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:05,228 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:05,228 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:05,228 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:05,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:05,229 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:05,229 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:05,230 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:05,230 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:05,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:05,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:05,232 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:05,235 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/E5F49419B1874726B4C92239FCE4DF2F/D60E1EDB895E44A88A3C4C0A8946DC9F/.tez/application_1463531944858_0001
2016-05-18 00:39:05,236 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false/_temporary
2016-05-18 00:39:05,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.399
2016-05-18 00:39:05,321 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:05,322 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:05,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 00:39:05,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-18 00:39:05,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:05,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:05,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:05,399 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:05,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false
2016-05-18 00:39:05,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:05,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:05,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-05-18 00:39:05,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:05,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:05,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:05,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...e/false_false_false_false
2016-05-18 00:39:05,463 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:05,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:05,482 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:05,482 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:05,482 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:05,491 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001 doesn't exist and is created
2016-05-18 00:39:05,506 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:05,506 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:05,508 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd
2016-05-18 00:39:05,520 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531945482_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:05,520 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:05,520 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:05,522 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:05,522 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:05,523 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:05,523 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:05,524 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:05,525 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:05,526 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:05,526 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:05,526 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:05,526 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:05,527 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:05,527 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:05,527 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:05,527 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:05,528 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:05,528 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:05,528 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:05,530 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:05,531 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:05,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:05,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-log-dir
2016-05-18 00:39:05,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-log-dir/history.txt.appattempt_1463531945482_0001_000000, maxErrors=10
2016-05-18 00:39:05,533 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:05,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531945482_0001_000000, appSubmitTime=1463531945519, launchTime=1463531945519
2016-05-18 00:39:05,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:05,545 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55721
2016-05-18 00:39:05,545 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531945482
2016-05-18 00:39:05,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531945482_0001_000000, startTime=1463531945547
2016-05-18 00:39:05,547 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:05,606 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:05,607 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:05,607 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1463531945482_0001, dagName=[2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false
2016-05-18 00:39:05,624 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false
2016-05-18 00:39:05,625 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:05,625 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531945482_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-log-dir/dag_1463531945482_0001_1.dot
2016-05-18 00:39:05,626 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-log-dir/dag_1463531945482_0001_1-tez-dag.pb.txt
2016-05-18 00:39:05,640 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531945482_0001_1
2016-05-18 00:39:05,640 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,640 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false
2016-05-18 00:39:05 Running Dag: dag_1463531945482_0001_1
2016-05-18 00:39:05,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531945482_0001_1, submitTime=1463531945625
2016-05-18 00:39:05,651 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001/tez-conf.pb" } size: 36903 timestamp: 1463531945000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:05,651 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,651 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:05,656 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001/tez-conf.pb" } size: 36903 timestamp: 1463531945000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:05,656 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,656 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:05,656 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:05,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531945482_0001_1, initTime=1463531945656
2016-05-18 00:39:05,658 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531945482_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:05,659 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:05,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:DAG_STARTED]: dagID=dag_1463531945482_0001_1, startTime=1463531945659
2016-05-18 00:39:05,659 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531945482_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:05,659 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A1AAE02597AB4ADEA69E259A99D31F21 : {21BACABCC5DF491CA1EABF00715B4A6B={InputName=21BACABCC5DF491CA1EABF00715B4A6B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:05,659 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 21BACABCC5DF491CA1EABF00715B4A6B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:05,659 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,660 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,660 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,660 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1463531945482_0001, dagName=[2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false
2016-05-18 00:39:05,660 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,661 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:05,661 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,661 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 21BACABCC5DF491CA1EABF00715B4A6B on vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,663 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 21BACABCC5DF491CA1EABF00715B4A6B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:05,664 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:05,664 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,665 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531945482_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:05,666 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2], numAdditionalOutputs=1
2016-05-18 00:39:05,666 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=ECA14FA6E29243249CD785210E5F2B0F, vertex=vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:05,667 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:05,669 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:05,670 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:05,670 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:05,671 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 21BACABCC5DF491CA1EABF00715B4A6B on vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,669 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for CD494676E4EB4AB38C8F12D79A94F3F2:ECA14FA6E29243249CD785210E5F2B0F using oldmapred API
2016-05-18 00:39:05,671 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=ECA14FA6E29243249CD785210E5F2B0F, vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:05,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, vertexId=vertex_1463531945482_0001_1_01, initRequestedTime=1463531945661, initedTime=1463531945666, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:05,673 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:05,674 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:05,674 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {21BACABCC5DF491CA1EABF00715B4A6B=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:05,676 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] parallelism set to 1
2016-05-18 00:39:05,676 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:05,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, vertexId=vertex_1463531945482_0001_1_00, initRequestedTime=1463531945659, initedTime=1463531945677, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:05,677 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] with distanceFromRoot: 0
2016-05-18 00:39:05,677 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:05,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531945482_0001_1_00, startRequestedTime=1463531945661, startedTime=1463531945678
2016-05-18 00:39:05,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531945482_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:05,679 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:05,679 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531945482_0001_1_00 for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] numStartedSources: 1 numSources: 1
2016-05-18 00:39:05,680 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] with distanceFromRoot: 1
2016-05-18 00:39:05,679 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A1AAE02597AB4ADEA69E259A99D31F21
2016-05-18 00:39:05,680 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531945482_0001_1_01, startRequestedTime=1463531945679, startedTime=1463531945680
2016-05-18 00:39:05,681 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:05,682 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: CD494676E4EB4AB38C8F12D79A94F3F2 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:05,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_STARTED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, taskId=task_1463531945482_0001_1_00_000000, scheduledTime=1463531945682, launchTime=1463531945682
2016-05-18 00:39:05,683 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A1AAE02597AB4ADEA69E259A99D31F21 in vertex: CD494676E4EB4AB38C8F12D79A94F3F2 numBipartiteSourceTasks: 1
2016-05-18 00:39:05,684 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,684 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:05,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531945482_0001_00_000001, launchTime=1463531945686
2016-05-18 00:39:05,687 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531945482_0001_00_000001
2016-05-18 00:39:05,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531945482_0001, with dag id: dag_1463531945482_0001_1
2016-05-18 00:39:05,688 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531945482_0001_1_00_000000_0] started. Is using containerId: [container_1463531945482_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:05,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0, startTime=1463531945688, containerId=container_1463531945482_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:05,689 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531945482_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05 Starting to run new task attempt: attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,690 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,690 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,690 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:05,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: A1AAE02597AB4ADEA69E259A99D31F21, VertexParallelism: 1, TaskAttemptID:attempt_1463531945482_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=21BACABCC5DF491CA1EABF00715B4A6B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CD494676E4EB4AB38C8F12D79A94F3F2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:05,691 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:05,693 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,699 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:05,703 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531945482_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:05,703 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531945482_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:05,704 INFO input.MRInput (MRInput.java:initialize(440)) - 21BACABCC5DF491CA1EABF00715B4A6B using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:05,704 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 21BACABCC5DF491CA1EABF00715B4A6B
2016-05-18 00:39:05,708 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1AAE02597AB4ADEA69E259A99D31F21, ordinal: 0
2016-05-18 00:39:05,709 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1AAE02597AB4ADEA69E259A99D31F21, mem on start (mb), free: 207, total: 338, max: 672
2016-05-18 00:39:05,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:05,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:05,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:05,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:05,711 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[21BACABCC5DF491CA1EABF00715B4A6B:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [CD494676E4EB4AB38C8F12D79A94F3F2:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:05,711 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:05,712 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:05,712 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:05,713 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[21BACABCC5DF491CA1EABF00715B4A6B:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [CD494676E4EB4AB38C8F12D79A94F3F2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:05,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:05,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 21BACABCC5DF491CA1EABF00715B4A6B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:05,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 21BACABCC5DF491CA1EABF00715B4A6B
2016-05-18 00:39:05,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:05,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:05,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:05,715 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,717 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:05,717 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 21BACABCC5DF491CA1EABF00715B4A6B initialized RecordReader from event
2016-05-18 00:39:05,718 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:05,719 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 21BACABCC5DF491CA1EABF00715B4A6B
2016-05-18 00:39:05,719 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: A82833A1C4DD486C8C342613B49C9731
2016-05-18 00:39:05,720 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A82833A1C4DD486C8C342613B49C9731
2016-05-18 00:39:05,722 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - CD494676E4EB4AB38C8F12D79A94F3F2 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@536ac63, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@486f9a41, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:05,723 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for CD494676E4EB4AB38C8F12D79A94F3F2: , UsingHashComparator=false
2016-05-18 00:39:05,743 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:05,744 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:05,744 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for CD494676E4EB4AB38C8F12D79A94F3F2: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:05,744 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - CD494676E4EB4AB38C8F12D79A94F3F2: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:05,745 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:05,745 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1AAE02597AB4ADEA69E259A99D31F21, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:05,750 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,751 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1AAE02597AB4ADEA69E259A99D31F21, mem on close (mb), free: 211, total: 338, max: 672
2016-05-18 00:39:05,751 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - CD494676E4EB4AB38C8F12D79A94F3F2: Starting flush of map output
2016-05-18 00:39:05,751 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - CD494676E4EB4AB38C8F12D79A94F3F2: Span0.length = 37, perItem = 24
2016-05-18 00:39:05,752 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - CD494676E4EB4AB38C8F12D79A94F3F2: done sorting span=0, length=37, time=1
2016-05-18 00:39:05,753 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - CD494676E4EB4AB38C8F12D79A94F3F2: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:05,754 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - CD494676E4EB4AB38C8F12D79A94F3F2: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-local-dir/output/attempt_1463531945482_0001_1_00_000000_0_10047_0/file.out
2016-05-18 00:39:05,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:05 Completed running task attempt: attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,758 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:05,758 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531945482_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:05,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531945482_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531945745, Process_Duration=5, Process_End_Time=1463531945750, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:05,760 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531945482_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,760 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, taskAttemptId=attempt_1463531945482_0001_1_00_000000_0, creationTime=1463531945683, allocationTime=1463531945685, startTime=1463531945688, finishTime=1463531945760, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:05,761 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531945482_0001_1_00_000000_0
2016-05-18 00:39:05,761 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:05,761 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,762 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531945482_0001_00_000001
2016-05-18 00:39:05,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_FINISHED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, taskId=task_1463531945482_0001_1_00_000000, startTime=1463531945688, finishTime=1463531945761, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1463531945482_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531945745, Process_Duration=5, Process_End_Time=1463531945750, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:05,762 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:05,762 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_FINISHED]: vertexName=A1AAE02597AB4ADEA69E259A99D31F21, vertexId=vertex_1463531945482_0001_1_00, initRequestedTime=1463531945659, initedTime=1463531945677, startRequestedTime=1463531945661, startedTime=1463531945678, finishTime=1463531945762, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531945745, Process_Duration=5, Process_End_Time=1463531945750, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531945688, firstTasksToStart=[ task_1463531945482_0001_1_00_000000 ], lastTaskFinishTime=1463531945760, lastTasksToFinish=[ task_1463531945482_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531945482_0001_1_00_000000 ], longestDurationTasks=[ task_1463531945482_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:05,764 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:05,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531945482_0001_00_000001, stoppedTime=1463531945764, exitStatus=0
2016-05-18 00:39:05,765 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] attempt: attempt_1463531945482_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:05,765 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:05,765 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CD494676E4EB4AB38C8F12D79A94F3F2
2016-05-18 00:39:05,765 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,765 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A1AAE02597AB4ADEA69E259A99D31F21, destinationVertexName=CD494676E4EB4AB38C8F12D79A94F3F2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:05,765 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,766 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531945482_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:05,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_STARTED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, taskId=task_1463531945482_0001_1_01_000000, scheduledTime=1463531945766, launchTime=1463531945766
2016-05-18 00:39:05,767 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531945482_0001_00_000002, launchTime=1463531945769
2016-05-18 00:39:05,769 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531945482_0001_00_000002
2016-05-18 00:39:05,770 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531945482_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,770 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531945482_0001_1_01_000000_0] started. Is using containerId: [container_1463531945482_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:05,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0, startTime=1463531945770, containerId=container_1463531945482_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:05 Starting to run new task attempt: attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,771 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,772 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,772 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531945482_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:05,772 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531945482_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:05,773 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531945482_0001_00_000001, Exiting
2016-05-18 00:39:05,773 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531945482_0001_00_000001
2016-05-18 00:39:05,773 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531945482_0001_00_000001
2016-05-18 00:39:05,773 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:05,773 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: CD494676E4EB4AB38C8F12D79A94F3F2, VertexParallelism: 1, TaskAttemptID:attempt_1463531945482_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A1AAE02597AB4ADEA69E259A99D31F21, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=ECA14FA6E29243249CD785210E5F2B0F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:05,774 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:05,774 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531945482_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:05,774 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,774 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531945482_0001_00_000001 completed successfully
2016-05-18 00:39:05,776 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531945482_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,777 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531945482_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:05,777 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:05,780 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:05,786 INFO output.MROutput (MROutput.java:initialize(421)) - ECA14FA6E29243249CD785210E5F2B0F: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:05,787 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CD494676E4EB4AB38C8F12D79A94F3F2, ordinal: 1
2016-05-18 00:39:05,787 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD494676E4EB4AB38C8F12D79A94F3F2, mem on start (mb), free: 206, total: 338, max: 672
2016-05-18 00:39:05,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:05,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:05,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:05,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:05,788 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A1AAE02597AB4ADEA69E259A99D31F21:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [ECA14FA6E29243249CD785210E5F2B0F:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:05,788 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:05,788 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:05,789 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:05,789 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A1AAE02597AB4ADEA69E259A99D31F21:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [ECA14FA6E29243249CD785210E5F2B0F:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:05,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:05,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A1AAE02597AB4ADEA69E259A99D31F21 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:05,790 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A1AAE02597AB4ADEA69E259A99D31F21: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:05,790 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A1AAE02597AB4ADEA69E259A99D31F21: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:05,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:05,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:05,791 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A1AAE02597AB4ADEA69E259A99D31F21 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:05,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A1AAE02597AB4ADEA69E259A99D31F21
2016-05-18 00:39:05,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:05,794 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,794 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1AAE02597AB4ADEA69E259A99D31F21: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:05,798 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531945482_0001_1_00_000000_0_10047} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531945798, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:05,798 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A1AAE02597AB4ADEA69E259A99D31F21
2016-05-18 00:39:05,798 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:05,799 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:05,799 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: A82833A1C4DD486C8C342613B49C9731
2016-05-18 00:39:05,799 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"], id: ECA14FA6E29243249CD785210E5F2B0F
2016-05-18 00:39:05,799 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-05-18 00:39:05,800 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A82833A1C4DD486C8C342613B49C9731, for 1 inputs
2016-05-18 00:39:05,799 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A1AAE02597AB4ADEA69E259A99D31F21, wasInterrupted=false
2016-05-18 00:39:05,800 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:05,801 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:05,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001_wd/localmode-local-dir/output/attempt_1463531945482_0001_1_00_000000_0_10047/file.out, len=732, isLocal=true
2016-05-18 00:39:05,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:05,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:05,802 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:05,803 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:05,803 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A1AAE02597AB4ADEA69E259A99D31F21
2016-05-18 00:39:05,803 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A1AAE02597AB4ADEA69E259A99D31F21: Shuffle Runner thread complete
2016-05-18 00:39:05,803 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CD494676E4EB4AB38C8F12D79A94F3F2, all 1 inputs ready in: 00:00:00.003
2016-05-18 00:39:05,804 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A1AAE02597AB4ADEA69E259A99D31F21: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:05,805 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:05,805 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:05,813 INFO output.MROutput (MROutput.java:close(526)) - ECA14FA6E29243249CD785210E5F2B0F closed
2016-05-18 00:39:05,813 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531945482_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:05,814 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,814 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD494676E4EB4AB38C8F12D79A94F3F2, mem on close (mb), free: 203, total: 338, max: 672
2016-05-18 00:39:05,814 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A1AAE02597AB4ADEA69E259A99D31F21
2016-05-18 00:39:05,814 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A1AAE02597AB4ADEA69E259A99D31F21: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:05,815 INFO output.MROutput (MROutput.java:close(526)) - ECA14FA6E29243249CD785210E5F2B0F closed
2016-05-18 00:39:05,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:05 Completed running task attempt: attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,815 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:05,815 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531945482_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:05,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531945482_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531945803, Process_Duration=11, Process_End_Time=1463531945814, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:05,818 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531945482_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,818 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:05,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, taskAttemptId=attempt_1463531945482_0001_1_01_000000_0, creationTime=1463531945767, allocationTime=1463531945768, startTime=1463531945770, finishTime=1463531945818, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:05,818 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531945482_0001_1_01_000000_0
2016-05-18 00:39:05,818 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:05,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:TASK_FINISHED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, taskId=task_1463531945482_0001_1_01_000000, startTime=1463531945770, finishTime=1463531945818, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463531945482_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531945803, Process_Duration=11, Process_End_Time=1463531945814, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:05,819 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:05,820 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2]
2016-05-18 00:39:05,819 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531945482_0001_00_000002
2016-05-18 00:39:05,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:VERTEX_FINISHED]: vertexName=CD494676E4EB4AB38C8F12D79A94F3F2, vertexId=vertex_1463531945482_0001_1_01, initRequestedTime=1463531945661, initedTime=1463531945666, startRequestedTime=1463531945679, startedTime=1463531945680, finishTime=1463531945820, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531945803, Process_Duration=11, Process_End_Time=1463531945814, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531945770, firstTasksToStart=[ task_1463531945482_0001_1_01_000000 ], lastTaskFinishTime=1463531945818, lastTasksToFinish=[ task_1463531945482_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531945482_0001_1_01_000000 ], longestDurationTasks=[ task_1463531945482_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:05,821 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:05,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531945482_0001_00_000002, stoppedTime=1463531945822, exitStatus=0
2016-05-18 00:39:05,822 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:05,822 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,822 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531945482_0001_1_00 [A1AAE02597AB4ADEA69E259A99D31F21]
2016-05-18 00:39:05,822 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531945482_0001_1
2016-05-18 00:39:05,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531945482_0001_1
2016-05-18 00:39:05,823 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: ECA14FA6E29243249CD785210E5F2B0F for vertex: vertex_1463531945482_0001_1_01 [CD494676E4EB4AB38C8F12D79A94F3F2], outputName: ECA14FA6E29243249CD785210E5F2B0F
2016-05-18 00:39:05,824 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531945482_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:05,825 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531945482_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:05,830 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:ECA14FA6E29243249CD785210E5F2B0F of vertex/vertexGroup:CD494676E4EB4AB38C8F12D79A94F3F2 isVertexGroupOutput:false
2016-05-18 00:39:05,830 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:05,830 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531945482_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:05,830 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531945482_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:05,831 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531945482_0001_00_000002, Exiting
2016-05-18 00:39:05,831 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531945482_0001_00_000002
2016-05-18 00:39:05,831 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531945482_0001_00_000002
2016-05-18 00:39:05,831 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531945482_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:05,831 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531945482_0001_00_000002 completed successfully
2016-05-18 00:39:05,833 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531945482_0001_1, queueSize=0
2016-05-18 00:39:05,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531945482_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531945482_0001_1, startTime=1463531945659, finishTime=1463531945830, timeTaken=171, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=310, AM_GC_TIME_MILLIS=8, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063891548, Process_Duration=16, Process_End_Time=2927063891564, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:05,834 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531945482_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:05,834 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531945482_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:05 Completed Dag: dag_1463531945482_0001_1
2016-05-18 00:39:05,835 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531945482_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:05,835 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531945482_0001_1_post
2016-05-18 00:39:05,835 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:05,835 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:05,836 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:05,837 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false, with id=dag_1463531945482_0001_1
2016-05-18 00:39:05,837 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313] comprehensivesort/(1/1) ...e/false_false_false_false, with id=dag_1463531945482_0001_1
2016-05-18 00:39:05,841 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1463531945482_0001
2016-05-18 00:39:05,845 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:05,845 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:05,845 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:05,845 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:05,845 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:05,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:05,846 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:05,847 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:05,846 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:05,847 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:05,847 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:05,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:05,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:05,849 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:05,852 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/2D285611D7D248D8ACCC098931B6DA5F/ECBDE276B63747FA870BB2401D2CC313/.tez/application_1463531945482_0001
2016-05-18 00:39:05,853 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false/_temporary
2016-05-18 00:39:05,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.391
2016-05-18 00:39:05,933 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:05,933 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:05,975 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 00:39:05,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 00:39:05,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:05,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:05,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:06,008 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:06,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false
2016-05-18 00:39:06,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:06,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-05-18 00:39:06,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:06,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:06,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:06,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ve/true_false_false_false
2016-05-18 00:39:06,077 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:06,091 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:06,095 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:06,095 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:06,095 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:06,104 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001 doesn't exist and is created
2016-05-18 00:39:06,118 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:06,118 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:06,119 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd
2016-05-18 00:39:06,129 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531946095_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:06,130 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:06,130 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:06,131 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:06,131 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:06,132 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:06,132 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:06,133 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:06,134 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:06,134 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:06,134 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:06,135 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:06,135 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:06,135 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:06,135 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:06,136 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:06,136 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:06,136 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:06,136 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:06,137 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:06,138 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:06,139 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:06,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:06,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-log-dir
2016-05-18 00:39:06,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-log-dir/history.txt.appattempt_1463531946095_0001_000000, maxErrors=10
2016-05-18 00:39:06,140 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:06,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531946095_0001_000000, appSubmitTime=1463531946129, launchTime=1463531946129
2016-05-18 00:39:06,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:06,150 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55356
2016-05-18 00:39:06,150 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531946095
2016-05-18 00:39:06,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531946095_0001_000000, startTime=1463531946151
2016-05-18 00:39:06,152 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:06,219 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:06,219 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:06,219 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1463531946095_0001, dagName=[17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-05-18 00:39:06,236 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-05-18 00:39:06,237 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:06,237 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531946095_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-log-dir/dag_1463531946095_0001_1.dot
2016-05-18 00:39:06,238 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-log-dir/dag_1463531946095_0001_1-tez-dag.pb.txt
2016-05-18 00:39:06,245 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531946095_0001_1
2016-05-18 00:39:06,245 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,245 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-05-18 00:39:06 Running Dag: dag_1463531946095_0001_1
2016-05-18 00:39:06,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531946095_0001_1, submitTime=1463531946236
2016-05-18 00:39:06,256 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001/tez-conf.pb" } size: 36903 timestamp: 1463531946000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:06,256 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,256 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:06,257 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001/tez-conf.pb" } size: 36903 timestamp: 1463531946000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:06,258 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,258 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:06,258 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:06,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531946095_0001_1, initTime=1463531946259
2016-05-18 00:39:06,259 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946095_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:06,261 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:06,261 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1463531946095_0001, dagName=[17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-05-18 00:39:06,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:DAG_STARTED]: dagID=dag_1463531946095_0001_1, startTime=1463531946261
2016-05-18 00:39:06,261 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946095_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:06,261 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B1A1A7F9BC7D4EECA5377F0E6B78BE8D : {98CAD49ED2C9412E9BD7C67E8E76FF75={InputName=98CAD49ED2C9412E9BD7C67E8E76FF75}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:06,261 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 98CAD49ED2C9412E9BD7C67E8E76FF75, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:06,261 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,262 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,262 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,262 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,262 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:06,263 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,263 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 98CAD49ED2C9412E9BD7C67E8E76FF75 on vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,266 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:06,266 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,266 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531946095_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:06,268 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871], numAdditionalOutputs=1
2016-05-18 00:39:06,268 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=8EBE671B073F4E5FB5FA638ECA144781, vertex=vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:06,268 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 98CAD49ED2C9412E9BD7C67E8E76FF75 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:06,271 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AA1D09E198D94919968B54B0A5919871:8EBE671B073F4E5FB5FA638ECA144781 using oldmapred API
2016-05-18 00:39:06,271 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=8EBE671B073F4E5FB5FA638ECA144781, vertexName=AA1D09E198D94919968B54B0A5919871, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:06,271 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:06,272 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:06,273 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:06,273 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:06,273 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 98CAD49ED2C9412E9BD7C67E8E76FF75 on vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AA1D09E198D94919968B54B0A5919871, vertexId=vertex_1463531946095_0001_1_01, initRequestedTime=1463531946263, initedTime=1463531946268, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:06,274 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:06,277 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:06,277 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {98CAD49ED2C9412E9BD7C67E8E76FF75=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:06,278 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] parallelism set to 1
2016-05-18 00:39:06,278 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:06,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, vertexId=vertex_1463531946095_0001_1_00, initRequestedTime=1463531946261, initedTime=1463531946279, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:06,279 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] with distanceFromRoot: 0
2016-05-18 00:39:06,279 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:06,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531946095_0001_1_00, startRequestedTime=1463531946263, startedTime=1463531946280
2016-05-18 00:39:06,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531946095_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:06,281 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:06,281 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531946095_0001_1_00 for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] numStartedSources: 1 numSources: 1
2016-05-18 00:39:06,281 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] with distanceFromRoot: 1
2016-05-18 00:39:06,281 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B1A1A7F9BC7D4EECA5377F0E6B78BE8D
2016-05-18 00:39:06,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531946095_0001, with dag id: dag_1463531946095_0001_1
2016-05-18 00:39:06,282 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531946095_0001_1_01, startRequestedTime=1463531946281, startedTime=1463531946282
2016-05-18 00:39:06,286 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:06,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_STARTED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, taskId=task_1463531946095_0001_1_00_000000, scheduledTime=1463531946286, launchTime=1463531946286
2016-05-18 00:39:06,287 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,286 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: AA1D09E198D94919968B54B0A5919871 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:06,290 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:06,291 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B1A1A7F9BC7D4EECA5377F0E6B78BE8D in vertex: AA1D09E198D94919968B54B0A5919871 numBipartiteSourceTasks: 1
2016-05-18 00:39:06,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531946095_0001_00_000001, launchTime=1463531946293
2016-05-18 00:39:06,294 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946095_0001_00_000001
2016-05-18 00:39:06,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531946095_0001_1_00_000000_0] started. Is using containerId: [container_1463531946095_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:06,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0, startTime=1463531946294, containerId=container_1463531946095_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:06,295 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946095_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06 Starting to run new task attempt: attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,296 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,296 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,296 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:06,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, VertexParallelism: 1, TaskAttemptID:attempt_1463531946095_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=98CAD49ED2C9412E9BD7C67E8E76FF75, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AA1D09E198D94919968B54B0A5919871, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:06,297 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:06,297 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,302 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:06,303 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531946095_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:06,303 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531946095_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:06,304 INFO input.MRInput (MRInput.java:initialize(440)) - 98CAD49ED2C9412E9BD7C67E8E76FF75 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:06,304 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 98CAD49ED2C9412E9BD7C67E8E76FF75
2016-05-18 00:39:06,310 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, ordinal: 0
2016-05-18 00:39:06,310 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, mem on start (mb), free: 105, total: 338, max: 672
2016-05-18 00:39:06,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:06,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:06,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:06,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:06,311 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[98CAD49ED2C9412E9BD7C67E8E76FF75:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AA1D09E198D94919968B54B0A5919871:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:06,311 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:06,312 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:06,312 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:06,312 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[98CAD49ED2C9412E9BD7C67E8E76FF75:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AA1D09E198D94919968B54B0A5919871:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:06,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:06,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 98CAD49ED2C9412E9BD7C67E8E76FF75 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:06,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:06,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:06,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 98CAD49ED2C9412E9BD7C67E8E76FF75
2016-05-18 00:39:06,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:06,314 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,315 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:06,315 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 98CAD49ED2C9412E9BD7C67E8E76FF75 initialized RecordReader from event
2016-05-18 00:39:06,317 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,318 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 98CAD49ED2C9412E9BD7C67E8E76FF75
2016-05-18 00:39:06,318 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: A6D83EAEA8024459A7970141748B4B82
2016-05-18 00:39:06,318 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6D83EAEA8024459A7970141748B4B82
2016-05-18 00:39:06,321 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AA1D09E198D94919968B54B0A5919871 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4481a83d, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@1c4c5fee, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:06,322 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AA1D09E198D94919968B54B0A5919871: , UsingHashComparator=false
2016-05-18 00:39:06,344 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:06,345 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:06,345 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AA1D09E198D94919968B54B0A5919871: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:06,345 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AA1D09E198D94919968B54B0A5919871: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:06,346 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,347 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:06,353 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,353 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, mem on close (mb), free: 209, total: 339, max: 672
2016-05-18 00:39:06,353 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AA1D09E198D94919968B54B0A5919871: Starting flush of map output
2016-05-18 00:39:06,354 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AA1D09E198D94919968B54B0A5919871: Span0.length = 37, perItem = 24
2016-05-18 00:39:06,356 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AA1D09E198D94919968B54B0A5919871: done sorting span=0, length=37, time=2
2016-05-18 00:39:06,356 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AA1D09E198D94919968B54B0A5919871: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:06,357 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AA1D09E198D94919968B54B0A5919871: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-local-dir/output/attempt_1463531946095_0001_1_00_000000_0_10053_0/file.out
2016-05-18 00:39:06,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:06 Completed running task attempt: attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,362 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:06,362 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531946095_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:06,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531946095_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531946347, Process_Duration=6, Process_End_Time=1463531946353, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:06,364 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531946095_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,364 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,364 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:06,364 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,364 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946095_0001_00_000001
2016-05-18 00:39:06,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, taskAttemptId=attempt_1463531946095_0001_1_00_000000_0, creationTime=1463531946287, allocationTime=1463531946293, startTime=1463531946294, finishTime=1463531946365, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:06,365 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531946095_0001_1_00_000000_0
2016-05-18 00:39:06,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_FINISHED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, taskId=task_1463531946095_0001_1_00_000000, startTime=1463531946294, finishTime=1463531946365, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463531946095_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531946347, Process_Duration=6, Process_End_Time=1463531946353, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:06,366 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:06,367 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_FINISHED]: vertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, vertexId=vertex_1463531946095_0001_1_00, initRequestedTime=1463531946261, initedTime=1463531946279, startRequestedTime=1463531946263, startedTime=1463531946280, finishTime=1463531946367, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531946347, Process_Duration=6, Process_End_Time=1463531946353, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531946294, firstTasksToStart=[ task_1463531946095_0001_1_00_000000 ], lastTaskFinishTime=1463531946365, lastTasksToFinish=[ task_1463531946095_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531946095_0001_1_00_000000 ], longestDurationTasks=[ task_1463531946095_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:06,368 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:06,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531946095_0001_00_000001, stoppedTime=1463531946368, exitStatus=0
2016-05-18 00:39:06,369 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] attempt: attempt_1463531946095_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:06,369 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AA1D09E198D94919968B54B0A5919871
2016-05-18 00:39:06,369 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, destinationVertexName=AA1D09E198D94919968B54B0A5919871. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:06,369 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,369 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:06,370 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:06,370 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531946095_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:06,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_STARTED]: vertexName=AA1D09E198D94919968B54B0A5919871, taskId=task_1463531946095_0001_1_01_000000, scheduledTime=1463531946370, launchTime=1463531946370
2016-05-18 00:39:06,371 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531946095_0001_00_000002, launchTime=1463531946372
2016-05-18 00:39:06,373 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946095_0001_00_000002
2016-05-18 00:39:06,374 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946095_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,374 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531946095_0001_1_01_000000_0] started. Is using containerId: [container_1463531946095_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:06,375 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531946095_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:06 Starting to run new task attempt: attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,375 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946095_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:06,375 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,375 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,375 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531946095_0001_00_000001, Exiting
2016-05-18 00:39:06,376 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:06,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AA1D09E198D94919968B54B0A5919871, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0, startTime=1463531946374, containerId=container_1463531946095_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:06,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: AA1D09E198D94919968B54B0A5919871, VertexParallelism: 1, TaskAttemptID:attempt_1463531946095_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1A1A7F9BC7D4EECA5377F0E6B78BE8D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8EBE671B073F4E5FB5FA638ECA144781, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:06,376 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531946095_0001_00_000001
2016-05-18 00:39:06,377 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:06,377 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531946095_0001_00_000001
2016-05-18 00:39:06,378 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,378 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531946095_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531946095_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:06,379 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531946095_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:06,380 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531946095_0001_00_000001 completed successfully
2016-05-18 00:39:06,382 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:06,385 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:06,392 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AA1D09E198D94919968B54B0A5919871, ordinal: 1
2016-05-18 00:39:06,392 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA1D09E198D94919968B54B0A5919871, mem on start (mb), free: 204, total: 339, max: 672
2016-05-18 00:39:06,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:06,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:06,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:06,397 INFO output.MROutput (MROutput.java:initialize(421)) - 8EBE671B073F4E5FB5FA638ECA144781: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:06,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:06,397 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B1A1A7F9BC7D4EECA5377F0E6B78BE8D:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [8EBE671B073F4E5FB5FA638ECA144781:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:06,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:06,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:06,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:06,398 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B1A1A7F9BC7D4EECA5377F0E6B78BE8D:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [8EBE671B073F4E5FB5FA638ECA144781:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:06,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:06,399 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:06,399 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:06,400 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B1A1A7F9BC7D4EECA5377F0E6B78BE8D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:06,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B1A1A7F9BC7D4EECA5377F0E6B78BE8D
2016-05-18 00:39:06,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B1A1A7F9BC7D4EECA5377F0E6B78BE8D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:06,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:06,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:06,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:06,403 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,403 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:06,405 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531946095_0001_1_00_000000_0_10053} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531946405, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:06,405 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B1A1A7F9BC7D4EECA5377F0E6B78BE8D
2016-05-18 00:39:06,405 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:06,406 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B1A1A7F9BC7D4EECA5377F0E6B78BE8D, wasInterrupted=false
2016-05-18 00:39:06,406 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:06,406 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:06,407 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:06,407 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: A6D83EAEA8024459A7970141748B4B82
2016-05-18 00:39:06,407 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"], id: 8EBE671B073F4E5FB5FA638ECA144781
2016-05-18 00:39:06,408 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-05-18 00:39:06,408 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6D83EAEA8024459A7970141748B4B82, for 1 inputs
2016-05-18 00:39:06,408 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001_wd/localmode-local-dir/output/attempt_1463531946095_0001_1_00_000000_0_10053/file.out, len=732, isLocal=true
2016-05-18 00:39:06,408 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:06,408 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:06,408 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:06,409 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:06,409 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B1A1A7F9BC7D4EECA5377F0E6B78BE8D
2016-05-18 00:39:06,409 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: Shuffle Runner thread complete
2016-05-18 00:39:06,410 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AA1D09E198D94919968B54B0A5919871, all 1 inputs ready in: 00:00:00.002
2016-05-18 00:39:06,411 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:06,417 INFO output.MROutput (MROutput.java:close(526)) - 8EBE671B073F4E5FB5FA638ECA144781 closed
2016-05-18 00:39:06,417 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531946095_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:06,418 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,418 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA1D09E198D94919968B54B0A5919871, mem on close (mb), free: 201, total: 339, max: 672
2016-05-18 00:39:06,418 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B1A1A7F9BC7D4EECA5377F0E6B78BE8D
2016-05-18 00:39:06,419 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1A1A7F9BC7D4EECA5377F0E6B78BE8D: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:06,419 INFO output.MROutput (MROutput.java:close(526)) - 8EBE671B073F4E5FB5FA638ECA144781 closed
2016-05-18 00:39:06,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:06 Completed running task attempt: attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,420 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:06,420 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531946095_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:06,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531946095_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531946410, Process_Duration=8, Process_End_Time=1463531946418, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:06,422 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531946095_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AA1D09E198D94919968B54B0A5919871, taskAttemptId=attempt_1463531946095_0001_1_01_000000_0, creationTime=1463531946371, allocationTime=1463531946372, startTime=1463531946374, finishTime=1463531946422, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:06,422 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,422 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:06,422 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,423 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946095_0001_00_000002
2016-05-18 00:39:06,422 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531946095_0001_1_01_000000_0
2016-05-18 00:39:06,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:TASK_FINISHED]: vertexName=AA1D09E198D94919968B54B0A5919871, taskId=task_1463531946095_0001_1_01_000000, startTime=1463531946374, finishTime=1463531946423, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463531946095_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531946410, Process_Duration=8, Process_End_Time=1463531946418, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:06,424 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:06,424 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871]
2016-05-18 00:39:06,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:VERTEX_FINISHED]: vertexName=AA1D09E198D94919968B54B0A5919871, vertexId=vertex_1463531946095_0001_1_01, initRequestedTime=1463531946263, initedTime=1463531946268, startRequestedTime=1463531946281, startedTime=1463531946282, finishTime=1463531946424, timeTaken=142, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531946410, Process_Duration=8, Process_End_Time=1463531946418, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531946374, firstTasksToStart=[ task_1463531946095_0001_1_01_000000 ], lastTaskFinishTime=1463531946422, lastTasksToFinish=[ task_1463531946095_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531946095_0001_1_01_000000 ], longestDurationTasks=[ task_1463531946095_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:06,426 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:06,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531946095_0001_00_000002, stoppedTime=1463531946426, exitStatus=0
2016-05-18 00:39:06,426 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:06,426 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:06,426 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531946095_0001_1_00 [B1A1A7F9BC7D4EECA5377F0E6B78BE8D]
2016-05-18 00:39:06,426 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531946095_0001_1
2016-05-18 00:39:06,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531946095_0001_1
2016-05-18 00:39:06,428 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 8EBE671B073F4E5FB5FA638ECA144781 for vertex: vertex_1463531946095_0001_1_01 [AA1D09E198D94919968B54B0A5919871], outputName: 8EBE671B073F4E5FB5FA638ECA144781
2016-05-18 00:39:06,428 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946095_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:06,429 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531946095_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:06,433 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531946095_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:06,433 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946095_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:06,433 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531946095_0001_00_000002, Exiting
2016-05-18 00:39:06,433 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531946095_0001_00_000002
2016-05-18 00:39:06,433 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531946095_0001_00_000002
2016-05-18 00:39:06,434 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531946095_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:06,434 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531946095_0001_00_000002 completed successfully
2016-05-18 00:39:06,434 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:8EBE671B073F4E5FB5FA638ECA144781 of vertex/vertexGroup:AA1D09E198D94919968B54B0A5919871 isVertexGroupOutput:false
2016-05-18 00:39:06,434 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:06,436 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531946095_0001_1, queueSize=0
2016-05-18 00:39:06,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946095_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531946095_0001_1, startTime=1463531946261, finishTime=1463531946434, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=320, AM_GC_TIME_MILLIS=9, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063892757, Process_Duration=14, Process_End_Time=2927063892771, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:06,437 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531946095_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:06,437 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946095_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:06 Completed Dag: dag_1463531946095_0001_1
2016-05-18 00:39:06,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531946095_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:06,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531946095_0001_1_post
2016-05-18 00:39:06,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:06,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:06,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false, with id=dag_1463531946095_0001_1
2016-05-18 00:39:06,439 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A] comprehensivesort/(1/1) ...ve/true_false_false_false, with id=dag_1463531946095_0001_1
2016-05-18 00:39:06,444 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1463531946095_0001
2016-05-18 00:39:06,446 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:06,446 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:06,447 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:06,447 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:06,447 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:06,447 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:06,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:06,448 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:06,448 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:06,448 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:06,448 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:06,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:06,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:06,449 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:06,452 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/17EC15E295D443E18BBEA20856D6689A/CCA6667A0C934480BA0D74B5A964BC9A/.tez/application_1463531946095_0001
2016-05-18 00:39:06,453 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false/_temporary
2016-05-18 00:39:06,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.378
2016-05-18 00:39:06,526 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:06,526 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:06,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 00:39:06,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 00:39:06,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:06,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:06,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:06,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false
2016-05-18 00:39:06,659 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:06,659 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,660 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-05-18 00:39:06,660 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:06,660 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:06,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:06,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/true_true_false_false
2016-05-18 00:39:06,662 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:06,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:06,679 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:06,679 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:06,680 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:06,688 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001 doesn't exist and is created
2016-05-18 00:39:06,703 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:06,703 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:06,704 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd
2016-05-18 00:39:06,713 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531946679_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:06,714 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:06,714 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:06,715 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:06,715 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:06,716 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:06,716 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:06,717 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:06,718 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:06,718 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:06,719 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:06,719 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:06,719 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:06,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:06,719 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:06,720 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:06,720 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:06,720 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:06,721 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:06,721 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:06,722 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:06,723 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:06,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:06,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-log-dir
2016-05-18 00:39:06,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-log-dir/history.txt.appattempt_1463531946679_0001_000000, maxErrors=10
2016-05-18 00:39:06,724 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:06,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531946679_0001_000000, appSubmitTime=1463531946713, launchTime=1463531946713
2016-05-18 00:39:06,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:06,735 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47443
2016-05-18 00:39:06,735 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531946679
2016-05-18 00:39:06,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531946679_0001_000000, startTime=1463531946737
2016-05-18 00:39:06,737 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:06,803 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:06,803 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:06,803 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1463531946679_0001, dagName=[735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-05-18 00:39:06,820 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-05-18 00:39:06,820 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:06,821 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531946679_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-log-dir/dag_1463531946679_0001_1.dot
2016-05-18 00:39:06,822 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-log-dir/dag_1463531946679_0001_1-tez-dag.pb.txt
2016-05-18 00:39:06,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531946679_0001_1
2016-05-18 00:39:06,829 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,829 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-05-18 00:39:06 Running Dag: dag_1463531946679_0001_1
2016-05-18 00:39:06,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531946679_0001_1, submitTime=1463531946820
2016-05-18 00:39:06,839 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001/tez-conf.pb" } size: 36903 timestamp: 1463531946000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:06,840 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,840 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:06,841 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001/tez-conf.pb" } size: 36903 timestamp: 1463531946000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:06,841 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,841 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:06,842 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:06,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531946679_0001_1, initTime=1463531946842
2016-05-18 00:39:06,843 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946679_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:06,843 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:06,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:DAG_STARTED]: dagID=dag_1463531946679_0001_1, startTime=1463531946843
2016-05-18 00:39:06,844 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946679_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C34B229E5C7743639DBD5C1AEE90ADBC : {662760B1274941ABB6D90DA16071C2E2={InputName=662760B1274941ABB6D90DA16071C2E2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 662760B1274941ABB6D90DA16071C2E2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,844 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,845 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1463531946679_0001, dagName=[735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-05-18 00:39:06,845 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:06,845 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,846 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 662760B1274941ABB6D90DA16071C2E2 on vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,848 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:06,849 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,849 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,849 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 662760B1274941ABB6D90DA16071C2E2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:06,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531946679_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:06,850 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779], numAdditionalOutputs=1
2016-05-18 00:39:06,852 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=9906B4403A114959ADDC0FB0AF439351, vertex=vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:06,853 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BA4F86C349194FA4AF2A7F4378943779:9906B4403A114959ADDC0FB0AF439351 using oldmapred API
2016-05-18 00:39:06,853 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=9906B4403A114959ADDC0FB0AF439351, vertexName=BA4F86C349194FA4AF2A7F4378943779, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:06,854 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:06,854 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:06,854 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:06,855 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:06,855 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 662760B1274941ABB6D90DA16071C2E2 on vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, vertexId=vertex_1463531946679_0001_1_01, initRequestedTime=1463531946845, initedTime=1463531946850, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:06,856 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:06,857 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:06,857 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {662760B1274941ABB6D90DA16071C2E2=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:06,858 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] parallelism set to 1
2016-05-18 00:39:06,858 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:06,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, vertexId=vertex_1463531946679_0001_1_00, initRequestedTime=1463531946844, initedTime=1463531946858, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:06,859 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] with distanceFromRoot: 0
2016-05-18 00:39:06,860 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:06,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531946679_0001_1_00, startRequestedTime=1463531946845, startedTime=1463531946860
2016-05-18 00:39:06,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531946679_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:06,862 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:06,862 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531946679_0001_1_00 for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] numStartedSources: 1 numSources: 1
2016-05-18 00:39:06,862 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] with distanceFromRoot: 1
2016-05-18 00:39:06,862 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C34B229E5C7743639DBD5C1AEE90ADBC
2016-05-18 00:39:06,863 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531946679_0001_1_01, startRequestedTime=1463531946862, startedTime=1463531946863
2016-05-18 00:39:06,865 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:06,866 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: BA4F86C349194FA4AF2A7F4378943779 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:06,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_STARTED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, taskId=task_1463531946679_0001_1_00_000000, scheduledTime=1463531946866, launchTime=1463531946866
2016-05-18 00:39:06,867 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: C34B229E5C7743639DBD5C1AEE90ADBC in vertex: BA4F86C349194FA4AF2A7F4378943779 numBipartiteSourceTasks: 1
2016-05-18 00:39:06,867 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,868 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:06,869 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531946679_0001, with dag id: dag_1463531946679_0001_1
2016-05-18 00:39:06,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531946679_0001_00_000001, launchTime=1463531946871
2016-05-18 00:39:06,872 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946679_0001_00_000001
2016-05-18 00:39:06,873 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531946679_0001_1_00_000000_0] started. Is using containerId: [container_1463531946679_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:06,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0, startTime=1463531946873, containerId=container_1463531946679_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:06,874 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946679_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06 Starting to run new task attempt: attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,876 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,876 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,876 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: C34B229E5C7743639DBD5C1AEE90ADBC, VertexParallelism: 1, TaskAttemptID:attempt_1463531946679_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=662760B1274941ABB6D90DA16071C2E2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA4F86C349194FA4AF2A7F4378943779, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:06,877 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:06,879 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531946679_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:06,879 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531946679_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:06,880 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,885 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:06,890 INFO input.MRInput (MRInput.java:initialize(440)) - 662760B1274941ABB6D90DA16071C2E2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:06,890 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 662760B1274941ABB6D90DA16071C2E2
2016-05-18 00:39:06,896 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C34B229E5C7743639DBD5C1AEE90ADBC, ordinal: 0
2016-05-18 00:39:06,896 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C34B229E5C7743639DBD5C1AEE90ADBC, mem on start (mb), free: 104, total: 339, max: 672
2016-05-18 00:39:06,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:06,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:06,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:06,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:06,897 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[662760B1274941ABB6D90DA16071C2E2:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [BA4F86C349194FA4AF2A7F4378943779:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:06,898 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:06,898 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:06,899 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:06,899 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[662760B1274941ABB6D90DA16071C2E2:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [BA4F86C349194FA4AF2A7F4378943779:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:06,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:06,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 662760B1274941ABB6D90DA16071C2E2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:06,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:06,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:06,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 662760B1274941ABB6D90DA16071C2E2
2016-05-18 00:39:06,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:06,903 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,905 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:06,905 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 662760B1274941ABB6D90DA16071C2E2 initialized RecordReader from event
2016-05-18 00:39:06,908 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,909 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 662760B1274941ABB6D90DA16071C2E2
2016-05-18 00:39:06,910 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 42C1C714E4244C2FBDEBF57E8DB40A97
2016-05-18 00:39:06,910 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 42C1C714E4244C2FBDEBF57E8DB40A97
2016-05-18 00:39:06,915 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - BA4F86C349194FA4AF2A7F4378943779 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@58738988, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@753fc41d, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:06,916 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for BA4F86C349194FA4AF2A7F4378943779: , UsingHashComparator=false
2016-05-18 00:39:06,939 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:06,941 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:06,941 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for BA4F86C349194FA4AF2A7F4378943779: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:06,942 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - BA4F86C349194FA4AF2A7F4378943779: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:06,942 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:06,943 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C34B229E5C7743639DBD5C1AEE90ADBC, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:06,948 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,948 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C34B229E5C7743639DBD5C1AEE90ADBC, mem on close (mb), free: 207, total: 340, max: 672
2016-05-18 00:39:06,948 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - BA4F86C349194FA4AF2A7F4378943779: Starting flush of map output
2016-05-18 00:39:06,949 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - BA4F86C349194FA4AF2A7F4378943779: Span0.length = 37, perItem = 24
2016-05-18 00:39:06,950 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - BA4F86C349194FA4AF2A7F4378943779: done sorting span=0, length=37, time=1
2016-05-18 00:39:06,951 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - BA4F86C349194FA4AF2A7F4378943779: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:06,954 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - BA4F86C349194FA4AF2A7F4378943779: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-local-dir/output/attempt_1463531946679_0001_1_00_000000_0_10059_0/file.out
2016-05-18 00:39:06,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:06 Completed running task attempt: attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,959 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:06,959 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531946679_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:06,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531946679_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531946943, Process_Duration=5, Process_End_Time=1463531946948, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:06,961 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531946679_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,962 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:06,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, taskAttemptId=attempt_1463531946679_0001_1_00_000000_0, creationTime=1463531946867, allocationTime=1463531946870, startTime=1463531946873, finishTime=1463531946961, timeTaken=88, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:06,962 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531946679_0001_1_00_000000_0
2016-05-18 00:39:06,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:06,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_FINISHED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, taskId=task_1463531946679_0001_1_00_000000, startTime=1463531946873, finishTime=1463531946962, timeTaken=89, status=SUCCEEDED, successfulAttemptID=attempt_1463531946679_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531946943, Process_Duration=5, Process_End_Time=1463531946948, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:06,963 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:06,963 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946679_0001_00_000001
2016-05-18 00:39:06,963 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:06,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_FINISHED]: vertexName=C34B229E5C7743639DBD5C1AEE90ADBC, vertexId=vertex_1463531946679_0001_1_00, initRequestedTime=1463531946844, initedTime=1463531946858, startRequestedTime=1463531946845, startedTime=1463531946860, finishTime=1463531946964, timeTaken=104, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531946943, Process_Duration=5, Process_End_Time=1463531946948, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531946873, firstTasksToStart=[ task_1463531946679_0001_1_00_000000 ], lastTaskFinishTime=1463531946961, lastTasksToFinish=[ task_1463531946679_0001_1_00_000000 ], minTaskDuration=88, maxTaskDuration=88, avgTaskDuration=88.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531946679_0001_1_00_000000 ], longestDurationTasks=[ task_1463531946679_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:06,966 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:06,966 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] attempt: attempt_1463531946679_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:06,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531946679_0001_00_000001, stoppedTime=1463531946966, exitStatus=0
2016-05-18 00:39:06,966 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:06,966 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:06,967 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531946679_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:06,968 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BA4F86C349194FA4AF2A7F4378943779
2016-05-18 00:39:06,968 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=C34B229E5C7743639DBD5C1AEE90ADBC, destinationVertexName=BA4F86C349194FA4AF2A7F4378943779. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:06,968 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_STARTED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, taskId=task_1463531946679_0001_1_01_000000, scheduledTime=1463531946969, launchTime=1463531946969
2016-05-18 00:39:06,970 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531946679_0001_00_000002, launchTime=1463531946971
2016-05-18 00:39:06,972 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946679_0001_00_000002
2016-05-18 00:39:06,974 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946679_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:06,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531946679_0001_1_01_000000_0] started. Is using containerId: [container_1463531946679_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:06,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0, startTime=1463531946973, containerId=container_1463531946679_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:06 Starting to run new task attempt: attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:06,975 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:06,976 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:06,976 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:06,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: BA4F86C349194FA4AF2A7F4378943779, VertexParallelism: 1, TaskAttemptID:attempt_1463531946679_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C34B229E5C7743639DBD5C1AEE90ADBC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9906B4403A114959ADDC0FB0AF439351, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:06,976 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:06,977 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:06,977 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531946679_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:06,977 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946679_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:06,977 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531946679_0001_00_000001, Exiting
2016-05-18 00:39:06,977 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531946679_0001_00_000001
2016-05-18 00:39:06,978 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531946679_0001_00_000001
2016-05-18 00:39:06,982 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:06,982 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531946679_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:06,983 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531946679_0001_00_000001 completed successfully
2016-05-18 00:39:06,985 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531946679_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:06,985 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531946679_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:06,987 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:07,001 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA4F86C349194FA4AF2A7F4378943779, ordinal: 1
2016-05-18 00:39:07,001 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA4F86C349194FA4AF2A7F4378943779, mem on start (mb), free: 202, total: 340, max: 672
2016-05-18 00:39:07,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:07,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:07,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:07,004 INFO output.MROutput (MROutput.java:initialize(421)) - 9906B4403A114959ADDC0FB0AF439351: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:07,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:07,005 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C34B229E5C7743639DBD5C1AEE90ADBC:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [9906B4403A114959ADDC0FB0AF439351:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:07,005 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:07,006 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:07,006 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:07,006 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C34B229E5C7743639DBD5C1AEE90ADBC:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [9906B4403A114959ADDC0FB0AF439351:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:07,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:07,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C34B229E5C7743639DBD5C1AEE90ADBC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:07,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:07,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:07,008 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - C34B229E5C7743639DBD5C1AEE90ADBC: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:07,008 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - C34B229E5C7743639DBD5C1AEE90ADBC: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:07,009 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: C34B229E5C7743639DBD5C1AEE90ADBC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:07,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C34B229E5C7743639DBD5C1AEE90ADBC
2016-05-18 00:39:07,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:07,011 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:07,012 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C34B229E5C7743639DBD5C1AEE90ADBC: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:07,016 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:07,019 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 42C1C714E4244C2FBDEBF57E8DB40A97
2016-05-18 00:39:07,019 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"], id: 9906B4403A114959ADDC0FB0AF439351
2016-05-18 00:39:07,015 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531946679_0001_1_00_000000_0_10059} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531947015, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:07,020 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : C34B229E5C7743639DBD5C1AEE90ADBC
2016-05-18 00:39:07,020 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:07,020 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: C34B229E5C7743639DBD5C1AEE90ADBC, wasInterrupted=false
2016-05-18 00:39:07,020 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-05-18 00:39:07,022 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 42C1C714E4244C2FBDEBF57E8DB40A97, for 1 inputs
2016-05-18 00:39:07,022 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:07,024 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:07,026 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001_wd/localmode-local-dir/output/attempt_1463531946679_0001_1_00_000000_0_10059/file.out, len=732, isLocal=true
2016-05-18 00:39:07,026 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:07,026 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:07,026 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:07,026 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:07,027 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA4F86C349194FA4AF2A7F4378943779, all 1 inputs ready in: 00:00:00.004
2016-05-18 00:39:07,027 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : C34B229E5C7743639DBD5C1AEE90ADBC
2016-05-18 00:39:07,028 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - C34B229E5C7743639DBD5C1AEE90ADBC: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:07,029 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - C34B229E5C7743639DBD5C1AEE90ADBC: Shuffle Runner thread complete
2016-05-18 00:39:07,034 INFO output.MROutput (MROutput.java:close(526)) - 9906B4403A114959ADDC0FB0AF439351 closed
2016-05-18 00:39:07,034 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:07,034 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:07,034 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531946679_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:07,036 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:07,036 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA4F86C349194FA4AF2A7F4378943779, mem on close (mb), free: 199, total: 340, max: 672
2016-05-18 00:39:07,036 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: C34B229E5C7743639DBD5C1AEE90ADBC
2016-05-18 00:39:07,037 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C34B229E5C7743639DBD5C1AEE90ADBC: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:07,037 INFO output.MROutput (MROutput.java:close(526)) - 9906B4403A114959ADDC0FB0AF439351 closed
2016-05-18 00:39:07,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:07 Completed running task attempt: attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:07,037 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:07,037 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531946679_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:07,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531946679_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531947027, Process_Duration=9, Process_End_Time=1463531947036, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:07,040 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531946679_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,040 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, taskAttemptId=attempt_1463531946679_0001_1_01_000000_0, creationTime=1463531946969, allocationTime=1463531946970, startTime=1463531946973, finishTime=1463531947040, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:07,040 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531946679_0001_1_01_000000_0
2016-05-18 00:39:07,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:TASK_FINISHED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, taskId=task_1463531946679_0001_1_01_000000, startTime=1463531946973, finishTime=1463531947040, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463531946679_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531947027, Process_Duration=9, Process_End_Time=1463531947036, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:07,041 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:07,041 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779]
2016-05-18 00:39:07,040 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:07,042 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,042 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531946679_0001_00_000002
2016-05-18 00:39:07,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:VERTEX_FINISHED]: vertexName=BA4F86C349194FA4AF2A7F4378943779, vertexId=vertex_1463531946679_0001_1_01, initRequestedTime=1463531946845, initedTime=1463531946850, startRequestedTime=1463531946862, startedTime=1463531946863, finishTime=1463531947042, timeTaken=179, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531947027, Process_Duration=9, Process_End_Time=1463531947036, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531946973, firstTasksToStart=[ task_1463531946679_0001_1_01_000000 ], lastTaskFinishTime=1463531947040, lastTasksToFinish=[ task_1463531946679_0001_1_01_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531946679_0001_1_01_000000 ], longestDurationTasks=[ task_1463531946679_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:07,043 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:07,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531946679_0001_00_000002, stoppedTime=1463531947044, exitStatus=0
2016-05-18 00:39:07,044 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:07,044 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:07,044 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531946679_0001_1_00 [C34B229E5C7743639DBD5C1AEE90ADBC]
2016-05-18 00:39:07,045 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531946679_0001_1
2016-05-18 00:39:07,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531946679_0001_1
2016-05-18 00:39:07,045 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946679_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:07,045 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 9906B4403A114959ADDC0FB0AF439351 for vertex: vertex_1463531946679_0001_1_01 [BA4F86C349194FA4AF2A7F4378943779], outputName: 9906B4403A114959ADDC0FB0AF439351
2016-05-18 00:39:07,045 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531946679_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:07,052 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:9906B4403A114959ADDC0FB0AF439351 of vertex/vertexGroup:BA4F86C349194FA4AF2A7F4378943779 isVertexGroupOutput:false
2016-05-18 00:39:07,052 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:07,053 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531946679_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:07,054 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531946679_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:07,054 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531946679_0001_1, queueSize=0
2016-05-18 00:39:07,054 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531946679_0001_00_000002, Exiting
2016-05-18 00:39:07,054 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531946679_0001_00_000002
2016-05-18 00:39:07,054 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531946679_0001_00_000002
2016-05-18 00:39:07,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531946679_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531946679_0001_1, startTime=1463531946843, finishTime=1463531947052, timeTaken=209, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=480, AM_GC_TIME_MILLIS=8, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063893970, Process_Duration=14, Process_End_Time=2927063893984, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=4, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:07,054 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531946679_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:07,054 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531946679_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:07 Completed Dag: dag_1463531946679_0001_1
2016-05-18 00:39:07,055 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531946679_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:07,055 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531946679_0001_1_post
2016-05-18 00:39:07,055 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,055 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:07,055 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:07,055 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false, with id=dag_1463531946679_0001_1
2016-05-18 00:39:07,055 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16] comprehensivesort/(1/1) ...ive/true_true_false_false, with id=dag_1463531946679_0001_1
2016-05-18 00:39:07,055 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531946679_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:07,055 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531946679_0001_00_000002 completed successfully
2016-05-18 00:39:07,067 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1463531946679_0001
2016-05-18 00:39:07,070 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:07,071 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:07,071 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:07,071 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:07,071 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:07,071 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:07,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:07,072 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:07,072 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:07,072 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:07,073 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:07,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:07,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:07,074 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:07,078 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/735D34B847D54EBE80AC8BCBDB8D5594/DD3DF80CFE334FE795C66AA66768DC16/.tez/application_1463531946679_0001
2016-05-18 00:39:07,078 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false/_temporary
2016-05-18 00:39:07,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.419
2016-05-18 00:39:07,173 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:07,174 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:07,213 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:39:07,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:39:07,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:07,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:07,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:07,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:07,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false
2016-05-18 00:39:07,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:07,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:07,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"]
2016-05-18 00:39:07,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:07,301 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:07,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:07,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ve/false_true_false_false
2016-05-18 00:39:07,302 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:07,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:07,324 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:07,325 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:07,325 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:07,336 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001 doesn't exist and is created
2016-05-18 00:39:07,350 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:07,350 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:07,351 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd
2016-05-18 00:39:07,362 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531947325_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:07,362 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:07,362 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:07,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:07,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:07,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:07,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:07,365 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:07,366 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:07,367 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:07,367 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:07,367 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:07,367 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:07,367 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:07,368 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:07,368 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:07,368 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:07,368 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:07,369 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:07,369 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:07,370 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:07,371 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:07,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:07,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-log-dir
2016-05-18 00:39:07,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-log-dir/history.txt.appattempt_1463531947325_0001_000000, maxErrors=10
2016-05-18 00:39:07,372 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:07,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531947325_0001_000000, appSubmitTime=1463531947361, launchTime=1463531947362
2016-05-18 00:39:07,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:07,381 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40584
2016-05-18 00:39:07,383 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531947325
2016-05-18 00:39:07,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531947325_0001_000000, startTime=1463531947386
2016-05-18 00:39:07,386 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:07,450 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:07,450 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:07,451 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1463531947325_0001, dagName=[82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-05-18 00:39:07,468 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-05-18 00:39:07,468 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:07,469 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531947325_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-log-dir/dag_1463531947325_0001_1.dot
2016-05-18 00:39:07,470 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-log-dir/dag_1463531947325_0001_1-tez-dag.pb.txt
2016-05-18 00:39:07,476 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531947325_0001_1
2016-05-18 00:39:07,476 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,476 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-05-18 00:39:07 Running Dag: dag_1463531947325_0001_1
2016-05-18 00:39:07,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531947325_0001_1, submitTime=1463531947468
2016-05-18 00:39:07,486 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001/tez-conf.pb" } size: 36903 timestamp: 1463531947000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:07,487 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,487 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:07,487 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001/tez-conf.pb" } size: 36903 timestamp: 1463531947000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:07,487 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,488 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:07,488 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:07,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531947325_0001_1, initTime=1463531947488
2016-05-18 00:39:07,489 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947325_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:07,489 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:07,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:DAG_STARTED]: dagID=dag_1463531947325_0001_1, startTime=1463531947489
2016-05-18 00:39:07,489 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947325_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:07,489 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A6DDC63ABAAF489DB1B6F6278DC4FC16 : {C5DE92A20B7442ED9B02529A096A23E3={InputName=C5DE92A20B7442ED9B02529A096A23E3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:07,489 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: C5DE92A20B7442ED9B02529A096A23E3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:07,489 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,490 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,490 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,490 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,491 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:07,491 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,491 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1463531947325_0001, dagName=[82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-05-18 00:39:07,493 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: C5DE92A20B7442ED9B02529A096A23E3 on vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,495 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input C5DE92A20B7442ED9B02529A096A23E3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:07,493 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:07,499 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:07,499 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,500 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531947325_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:07,501 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140], numAdditionalOutputs=1
2016-05-18 00:39:07,501 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=41792BBDE1694627B32BFB4673942554, vertex=vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:07,503 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A9C6525CB3214C49B834DB9A99D81140:41792BBDE1694627B32BFB4673942554 using oldmapred API
2016-05-18 00:39:07,504 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=41792BBDE1694627B32BFB4673942554, vertexName=A9C6525CB3214C49B834DB9A99D81140, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:07,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, vertexId=vertex_1463531947325_0001_1_01, initRequestedTime=1463531947491, initedTime=1463531947501, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:07,505 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:07,506 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:07,506 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:07,506 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:07,507 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: C5DE92A20B7442ED9B02529A096A23E3 on vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,509 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:07,510 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {C5DE92A20B7442ED9B02529A096A23E3=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:07,510 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] parallelism set to 1
2016-05-18 00:39:07,511 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:07,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, vertexId=vertex_1463531947325_0001_1_00, initRequestedTime=1463531947489, initedTime=1463531947511, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:07,511 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] with distanceFromRoot: 0
2016-05-18 00:39:07,511 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:07,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531947325_0001_1_00, startRequestedTime=1463531947491, startedTime=1463531947511
2016-05-18 00:39:07,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531947325_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:07,512 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:07,512 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531947325_0001_1_00 for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] numStartedSources: 1 numSources: 1
2016-05-18 00:39:07,512 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] with distanceFromRoot: 1
2016-05-18 00:39:07,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531947325_0001_1_01, startRequestedTime=1463531947512, startedTime=1463531947512
2016-05-18 00:39:07,517 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:07,518 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A6DDC63ABAAF489DB1B6F6278DC4FC16
2016-05-18 00:39:07,518 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_STARTED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, taskId=task_1463531947325_0001_1_00_000000, scheduledTime=1463531947518, launchTime=1463531947518
2016-05-18 00:39:07,519 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,520 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:07,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531947325_0001, with dag id: dag_1463531947325_0001_1
2016-05-18 00:39:07,528 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A9C6525CB3214C49B834DB9A99D81140 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:07,530 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A6DDC63ABAAF489DB1B6F6278DC4FC16 in vertex: A9C6525CB3214C49B834DB9A99D81140 numBipartiteSourceTasks: 1
2016-05-18 00:39:07,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531947325_0001_00_000001, launchTime=1463531947528
2016-05-18 00:39:07,537 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947325_0001_00_000001
2016-05-18 00:39:07,538 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531947325_0001_1_00_000000_0] started. Is using containerId: [container_1463531947325_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:07,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0, startTime=1463531947537, containerId=container_1463531947325_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:07,538 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947325_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07 Starting to run new task attempt: attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,539 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,539 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,539 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:07,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: A6DDC63ABAAF489DB1B6F6278DC4FC16, VertexParallelism: 1, TaskAttemptID:attempt_1463531947325_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C5DE92A20B7442ED9B02529A096A23E3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A9C6525CB3214C49B834DB9A99D81140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:07,540 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:07,543 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531947325_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:07,543 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531947325_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:07,543 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,550 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:07,554 INFO input.MRInput (MRInput.java:initialize(440)) - C5DE92A20B7442ED9B02529A096A23E3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:07,554 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: C5DE92A20B7442ED9B02529A096A23E3
2016-05-18 00:39:07,557 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6DDC63ABAAF489DB1B6F6278DC4FC16, ordinal: 0
2016-05-18 00:39:07,557 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6DDC63ABAAF489DB1B6F6278DC4FC16, mem on start (mb), free: 98, total: 340, max: 672
2016-05-18 00:39:07,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:07,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:07,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:07,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:07,558 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[C5DE92A20B7442ED9B02529A096A23E3:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A9C6525CB3214C49B834DB9A99D81140:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:07,559 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:07,559 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:07,560 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:07,560 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[C5DE92A20B7442ED9B02529A096A23E3:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A9C6525CB3214C49B834DB9A99D81140:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:07,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:07,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C5DE92A20B7442ED9B02529A096A23E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:07,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C5DE92A20B7442ED9B02529A096A23E3
2016-05-18 00:39:07,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:07,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:07,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:07,562 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,565 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:07,566 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: C5DE92A20B7442ED9B02529A096A23E3
2016-05-18 00:39:07,566 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: EED17D84B3E94B2AACED690495A8317C
2016-05-18 00:39:07,566 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] EED17D84B3E94B2AACED690495A8317C
2016-05-18 00:39:07,569 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A9C6525CB3214C49B834DB9A99D81140 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4fbeb852, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@72373864, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:07,569 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A9C6525CB3214C49B834DB9A99D81140: , UsingHashComparator=false
2016-05-18 00:39:07,571 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:07,571 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - C5DE92A20B7442ED9B02529A096A23E3 initialized RecordReader from event
2016-05-18 00:39:07,594 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:07,594 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:07,594 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A9C6525CB3214C49B834DB9A99D81140: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:07,595 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A9C6525CB3214C49B834DB9A99D81140: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:07,595 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:07,596 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6DDC63ABAAF489DB1B6F6278DC4FC16, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:07,603 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,603 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6DDC63ABAAF489DB1B6F6278DC4FC16, mem on close (mb), free: 205, total: 341, max: 672
2016-05-18 00:39:07,604 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A9C6525CB3214C49B834DB9A99D81140: Starting flush of map output
2016-05-18 00:39:07,604 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A9C6525CB3214C49B834DB9A99D81140: Span0.length = 37, perItem = 24
2016-05-18 00:39:07,606 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A9C6525CB3214C49B834DB9A99D81140: done sorting span=0, length=37, time=2
2016-05-18 00:39:07,606 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A9C6525CB3214C49B834DB9A99D81140: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:07,608 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A9C6525CB3214C49B834DB9A99D81140: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-local-dir/output/attempt_1463531947325_0001_1_00_000000_0_10065_0/file.out
2016-05-18 00:39:07,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:07 Completed running task attempt: attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,612 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:07,612 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531947325_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:07,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531947325_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531947596, Process_Duration=7, Process_End_Time=1463531947603, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:07,614 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531947325_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,614 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, taskAttemptId=attempt_1463531947325_0001_1_00_000000_0, creationTime=1463531947519, allocationTime=1463531947526, startTime=1463531947537, finishTime=1463531947614, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:07,615 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531947325_0001_1_00_000000_0
2016-05-18 00:39:07,615 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:07,615 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,616 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947325_0001_00_000001
2016-05-18 00:39:07,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_FINISHED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, taskId=task_1463531947325_0001_1_00_000000, startTime=1463531947537, finishTime=1463531947615, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1463531947325_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531947596, Process_Duration=7, Process_End_Time=1463531947603, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:07,617 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:07,617 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_FINISHED]: vertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, vertexId=vertex_1463531947325_0001_1_00, initRequestedTime=1463531947489, initedTime=1463531947511, startRequestedTime=1463531947491, startedTime=1463531947511, finishTime=1463531947617, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531947596, Process_Duration=7, Process_End_Time=1463531947603, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531947537, firstTasksToStart=[ task_1463531947325_0001_1_00_000000 ], lastTaskFinishTime=1463531947614, lastTasksToFinish=[ task_1463531947325_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531947325_0001_1_00_000000 ], longestDurationTasks=[ task_1463531947325_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:07,618 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:07,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531947325_0001_00_000001, stoppedTime=1463531947618, exitStatus=0
2016-05-18 00:39:07,619 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] attempt: attempt_1463531947325_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:07,619 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:07,619 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A9C6525CB3214C49B834DB9A99D81140
2016-05-18 00:39:07,619 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:07,619 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, destinationVertexName=A9C6525CB3214C49B834DB9A99D81140. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:07,619 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_STARTED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, taskId=task_1463531947325_0001_1_01_000000, scheduledTime=1463531947620, launchTime=1463531947620
2016-05-18 00:39:07,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531947325_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:07,620 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531947325_0001_00_000002, launchTime=1463531947622
2016-05-18 00:39:07,623 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947325_0001_00_000002
2016-05-18 00:39:07,624 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531947325_0001_1_01_000000_0] started. Is using containerId: [container_1463531947325_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:07,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0, startTime=1463531947624, containerId=container_1463531947325_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:07,625 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947325_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07 Starting to run new task attempt: attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,625 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,626 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:07,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: A9C6525CB3214C49B834DB9A99D81140, VertexParallelism: 1, TaskAttemptID:attempt_1463531947325_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A6DDC63ABAAF489DB1B6F6278DC4FC16, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=41792BBDE1694627B32BFB4673942554, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:07,626 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:07,627 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531947325_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:07,627 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947325_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:07,627 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531947325_0001_00_000001, Exiting
2016-05-18 00:39:07,628 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531947325_0001_00_000001
2016-05-18 00:39:07,628 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531947325_0001_00_000001
2016-05-18 00:39:07,630 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,631 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531947325_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,632 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531947325_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:07,633 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531947325_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:07,633 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531947325_0001_00_000001 completed successfully
2016-05-18 00:39:07,636 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:07,640 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:07,649 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A9C6525CB3214C49B834DB9A99D81140, ordinal: 1
2016-05-18 00:39:07,650 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9C6525CB3214C49B834DB9A99D81140, mem on start (mb), free: 199, total: 341, max: 672
2016-05-18 00:39:07,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:07,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:07,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:07,652 INFO output.MROutput (MROutput.java:initialize(421)) - 41792BBDE1694627B32BFB4673942554: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:07,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:07,652 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A6DDC63ABAAF489DB1B6F6278DC4FC16:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [41792BBDE1694627B32BFB4673942554:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:07,653 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:07,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:07,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:07,654 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A6DDC63ABAAF489DB1B6F6278DC4FC16:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [41792BBDE1694627B32BFB4673942554:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:07,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:07,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A6DDC63ABAAF489DB1B6F6278DC4FC16 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:07,656 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:07,656 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:07,657 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A6DDC63ABAAF489DB1B6F6278DC4FC16 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:07,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A6DDC63ABAAF489DB1B6F6278DC4FC16
2016-05-18 00:39:07,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:07,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:07,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:07,660 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,662 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:07,663 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:07,664 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: EED17D84B3E94B2AACED690495A8317C
2016-05-18 00:39:07,664 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"], id: 41792BBDE1694627B32BFB4673942554
2016-05-18 00:39:07,664 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"]
2016-05-18 00:39:07,665 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] EED17D84B3E94B2AACED690495A8317C, for 1 inputs
2016-05-18 00:39:07,666 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531947325_0001_1_00_000000_0_10065} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531947666, TimeTaken=2, Rate=0.35 MB/s
2016-05-18 00:39:07,667 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A6DDC63ABAAF489DB1B6F6278DC4FC16
2016-05-18 00:39:07,667 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:07,667 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A6DDC63ABAAF489DB1B6F6278DC4FC16, wasInterrupted=false
2016-05-18 00:39:07,668 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:07,669 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:07,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001_wd/localmode-local-dir/output/attempt_1463531947325_0001_1_00_000000_0_10065/file.out, len=732, isLocal=true
2016-05-18 00:39:07,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:07,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:07,670 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:07,671 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:07,671 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A6DDC63ABAAF489DB1B6F6278DC4FC16
2016-05-18 00:39:07,671 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: Shuffle Runner thread complete
2016-05-18 00:39:07,672 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A9C6525CB3214C49B834DB9A99D81140, all 1 inputs ready in: 00:00:00.006
2016-05-18 00:39:07,673 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:07,679 INFO output.MROutput (MROutput.java:close(526)) - 41792BBDE1694627B32BFB4673942554 closed
2016-05-18 00:39:07,680 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531947325_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:07,681 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,681 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A9C6525CB3214C49B834DB9A99D81140, mem on close (mb), free: 197, total: 341, max: 672
2016-05-18 00:39:07,682 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A6DDC63ABAAF489DB1B6F6278DC4FC16
2016-05-18 00:39:07,682 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A6DDC63ABAAF489DB1B6F6278DC4FC16: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:07,683 INFO output.MROutput (MROutput.java:close(526)) - 41792BBDE1694627B32BFB4673942554 closed
2016-05-18 00:39:07,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:07 Completed running task attempt: attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,684 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:07,685 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531947325_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:07,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531947325_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531947672, Process_Duration=9, Process_End_Time=1463531947681, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:07,687 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531947325_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,687 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:07,687 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:07,687 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, taskAttemptId=attempt_1463531947325_0001_1_01_000000_0, creationTime=1463531947620, allocationTime=1463531947621, startTime=1463531947624, finishTime=1463531947688, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:07,688 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531947325_0001_1_01_000000_0
2016-05-18 00:39:07,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:TASK_FINISHED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, taskId=task_1463531947325_0001_1_01_000000, startTime=1463531947624, finishTime=1463531947688, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463531947325_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531947672, Process_Duration=9, Process_End_Time=1463531947681, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:07,689 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:07,689 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140]
2016-05-18 00:39:07,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:VERTEX_FINISHED]: vertexName=A9C6525CB3214C49B834DB9A99D81140, vertexId=vertex_1463531947325_0001_1_01, initRequestedTime=1463531947491, initedTime=1463531947501, startRequestedTime=1463531947512, startedTime=1463531947512, finishTime=1463531947689, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531947672, Process_Duration=9, Process_End_Time=1463531947681, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531947624, firstTasksToStart=[ task_1463531947325_0001_1_01_000000 ], lastTaskFinishTime=1463531947688, lastTasksToFinish=[ task_1463531947325_0001_1_01_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531947325_0001_1_01_000000 ], longestDurationTasks=[ task_1463531947325_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:07,691 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:07,691 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:07,691 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:07,691 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531947325_0001_1_00 [A6DDC63ABAAF489DB1B6F6278DC4FC16]
2016-05-18 00:39:07,691 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531947325_0001_1
2016-05-18 00:39:07,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531947325_0001_1
2016-05-18 00:39:07,692 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 41792BBDE1694627B32BFB4673942554 for vertex: vertex_1463531947325_0001_1_01 [A9C6525CB3214C49B834DB9A99D81140], outputName: 41792BBDE1694627B32BFB4673942554
2016-05-18 00:39:07,692 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947325_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:07,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531947325_0001_00_000002, stoppedTime=1463531947694, exitStatus=0
2016-05-18 00:39:07,694 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531947325_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:07,695 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947325_0001_00_000002
2016-05-18 00:39:07,695 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531947325_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:07,695 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947325_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:07,695 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531947325_0001_00_000002, Exiting
2016-05-18 00:39:07,695 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531947325_0001_00_000002
2016-05-18 00:39:07,695 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531947325_0001_00_000002
2016-05-18 00:39:07,696 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531947325_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:07,696 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531947325_0001_00_000002 completed successfully
2016-05-18 00:39:07,701 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:41792BBDE1694627B32BFB4673942554 of vertex/vertexGroup:A9C6525CB3214C49B834DB9A99D81140 isVertexGroupOutput:false
2016-05-18 00:39:07,701 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:07,704 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531947325_0001_1, queueSize=0
2016-05-18 00:39:07,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947325_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531947325_0001_1, startTime=1463531947489, finishTime=1463531947701, timeTaken=212, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=590, AM_GC_TIME_MILLIS=9, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063895268, Process_Duration=16, Process_End_Time=2927063895284, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:07,706 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531947325_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:07,706 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947325_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:07 Completed Dag: dag_1463531947325_0001_1
2016-05-18 00:39:07,706 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531947325_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:07,706 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531947325_0001_1_post
2016-05-18 00:39:07,706 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:07,706 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:07,706 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:07,707 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false, with id=dag_1463531947325_0001_1
2016-05-18 00:39:07,707 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8] comprehensivesort/(1/1) ...ve/false_true_false_false, with id=dag_1463531947325_0001_1
2016-05-18 00:39:07,716 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1463531947325_0001
2016-05-18 00:39:07,718 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:07,718 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:07,718 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:07,719 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:07,719 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:07,719 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:07,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:07,720 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:07,720 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:07,720 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:07,721 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:07,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:07,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:07,722 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:07,728 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/82381EF2B2704EFA9D45ABF4C471198F/ABCFE0804CDD4DCE9F929E75FA0C75A8/.tez/application_1463531947325_0001
2016-05-18 00:39:07,729 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false/_temporary
2016-05-18 00:39:07,730 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.428
2016-05-18 00:39:07,814 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:07,814 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:07,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 00:39:07,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-18 00:39:07,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:07,863 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:07,863 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:07,878 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:07,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false
2016-05-18 00:39:07,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:07,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:07,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"]
2016-05-18 00:39:07,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:07,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:07,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:07,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_true_false
2016-05-18 00:39:07,941 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:07,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:07,963 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:07,964 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:07,964 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:07,973 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001 doesn't exist and is created
2016-05-18 00:39:07,987 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:07,988 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:07,988 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd
2016-05-18 00:39:07,998 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531947963_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:07,999 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:07,999 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:08,000 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:08,000 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:08,001 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:08,001 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:08,002 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:08,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:08,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:08,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:08,003 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:08,004 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:08,004 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:08,004 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:08,004 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:08,005 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:08,005 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:08,005 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:08,005 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:08,007 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:08,007 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:08,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:08,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-log-dir
2016-05-18 00:39:08,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-log-dir/history.txt.appattempt_1463531947963_0001_000000, maxErrors=10
2016-05-18 00:39:08,009 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:08,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531947963_0001_000000, appSubmitTime=1463531947998, launchTime=1463531947998
2016-05-18 00:39:08,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:08,019 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56545
2016-05-18 00:39:08,021 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531947963
2016-05-18 00:39:08,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531947963_0001_000000, startTime=1463531948023
2016-05-18 00:39:08,023 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:08,088 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:08,088 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:08,088 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1463531947963_0001, dagName=[86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-05-18 00:39:08,107 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-05-18 00:39:08,107 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:08,108 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531947963_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-log-dir/dag_1463531947963_0001_1.dot
2016-05-18 00:39:08,108 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-log-dir/dag_1463531947963_0001_1-tez-dag.pb.txt
2016-05-18 00:39:08,114 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531947963_0001_1
2016-05-18 00:39:08,115 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,115 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-05-18 00:39:08 Running Dag: dag_1463531947963_0001_1
2016-05-18 00:39:08,124 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531947963_0001_1, submitTime=1463531948107
2016-05-18 00:39:08,126 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001/tez-conf.pb" } size: 36903 timestamp: 1463531947000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:08,126 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,127 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:08,128 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001/tez-conf.pb" } size: 36903 timestamp: 1463531947000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:08,128 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,128 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:08,128 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:08,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531947963_0001_1, initTime=1463531948130
2016-05-18 00:39:08,130 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947963_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:08,130 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:08,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:DAG_STARTED]: dagID=dag_1463531947963_0001_1, startTime=1463531948130
2016-05-18 00:39:08,131 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947963_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:08,131 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E4B70763FEA041B0AAEF0B774DF8864B : {F4F363CCF8524BB68646A24056B30155={InputName=F4F363CCF8524BB68646A24056B30155}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:08,131 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: F4F363CCF8524BB68646A24056B30155, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:08,131 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,131 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,131 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,132 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,132 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:08,132 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: F4F363CCF8524BB68646A24056B30155 on vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,132 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,134 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input F4F363CCF8524BB68646A24056B30155 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:08,135 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1463531947963_0001, dagName=[86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-05-18 00:39:08,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:08,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:08,136 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:08,137 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:08,137 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: F4F363CCF8524BB68646A24056B30155 on vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,138 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:08,138 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,139 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531947963_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:08,139 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5], numAdditionalOutputs=1
2016-05-18 00:39:08,140 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A0664B4A2B374E6F9219EA4A751635FE, vertex=vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:08,141 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D7B5BA69DB954A94BAD1677D15B3A6E5:A0664B4A2B374E6F9219EA4A751635FE using oldmapred API
2016-05-18 00:39:08,142 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A0664B4A2B374E6F9219EA4A751635FE, vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:08,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, vertexId=vertex_1463531947963_0001_1_01, initRequestedTime=1463531948132, initedTime=1463531948139, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:08,143 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:08,145 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:08,145 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {F4F363CCF8524BB68646A24056B30155=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:08,146 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] parallelism set to 1
2016-05-18 00:39:08,146 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:08,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, vertexId=vertex_1463531947963_0001_1_00, initRequestedTime=1463531948131, initedTime=1463531948147, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:08,147 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] with distanceFromRoot: 0
2016-05-18 00:39:08,147 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:08,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531947963_0001_1_00, startRequestedTime=1463531948132, startedTime=1463531948147
2016-05-18 00:39:08,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531947963_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:08,148 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:08,148 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531947963_0001_1_00 for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] numStartedSources: 1 numSources: 1
2016-05-18 00:39:08,148 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] with distanceFromRoot: 1
2016-05-18 00:39:08,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531947963_0001_1_01, startRequestedTime=1463531948148, startedTime=1463531948148
2016-05-18 00:39:08,148 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:08,150 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D7B5BA69DB954A94BAD1677D15B3A6E5 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:08,150 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E4B70763FEA041B0AAEF0B774DF8864B
2016-05-18 00:39:08,150 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E4B70763FEA041B0AAEF0B774DF8864B in vertex: D7B5BA69DB954A94BAD1677D15B3A6E5 numBipartiteSourceTasks: 1
2016-05-18 00:39:08,151 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_STARTED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, taskId=task_1463531947963_0001_1_00_000000, scheduledTime=1463531948151, launchTime=1463531948151
2016-05-18 00:39:08,152 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,153 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:08,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531947963_0001_00_000001, launchTime=1463531948154
2016-05-18 00:39:08,155 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947963_0001_00_000001
2016-05-18 00:39:08,156 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531947963_0001_1_00_000000_0] started. Is using containerId: [container_1463531947963_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:08,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0, startTime=1463531948156, containerId=container_1463531947963_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:08,157 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947963_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08 Starting to run new task attempt: attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,158 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531947963_0001, with dag id: dag_1463531947963_0001_1
2016-05-18 00:39:08,159 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:08,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: E4B70763FEA041B0AAEF0B774DF8864B, VertexParallelism: 1, TaskAttemptID:attempt_1463531947963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F4F363CCF8524BB68646A24056B30155, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:08,160 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:08,161 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,162 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531947963_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:08,162 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531947963_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:08,167 INFO input.MRInput (MRInput.java:initialize(440)) - F4F363CCF8524BB68646A24056B30155 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:08,168 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:08,168 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: F4F363CCF8524BB68646A24056B30155
2016-05-18 00:39:08,174 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4B70763FEA041B0AAEF0B774DF8864B, ordinal: 0
2016-05-18 00:39:08,174 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B70763FEA041B0AAEF0B774DF8864B, mem on start (mb), free: 100, total: 341, max: 672
2016-05-18 00:39:08,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:08,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:08,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:08,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:08,174 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F4F363CCF8524BB68646A24056B30155:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D7B5BA69DB954A94BAD1677D15B3A6E5:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:08,175 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:08,175 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:08,175 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:08,175 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F4F363CCF8524BB68646A24056B30155:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D7B5BA69DB954A94BAD1677D15B3A6E5:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:08,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:08,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F4F363CCF8524BB68646A24056B30155 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:08,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:08,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:08,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F4F363CCF8524BB68646A24056B30155
2016-05-18 00:39:08,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:08,177 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,178 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:08,178 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - F4F363CCF8524BB68646A24056B30155 initialized RecordReader from event
2016-05-18 00:39:08,179 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:08,180 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: F4F363CCF8524BB68646A24056B30155
2016-05-18 00:39:08,180 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 587265A775424EABB9DC2E4D45B25F76
2016-05-18 00:39:08,181 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 587265A775424EABB9DC2E4D45B25F76
2016-05-18 00:39:08,183 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D7B5BA69DB954A94BAD1677D15B3A6E5 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4ae5989f, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@27871523, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:08,183 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D7B5BA69DB954A94BAD1677D15B3A6E5: , UsingHashComparator=false
2016-05-18 00:39:08,203 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:08,203 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:08,204 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D7B5BA69DB954A94BAD1677D15B3A6E5: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:08,204 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D7B5BA69DB954A94BAD1677D15B3A6E5: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:08,204 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:08,205 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4B70763FEA041B0AAEF0B774DF8864B, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:08,210 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,210 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B70763FEA041B0AAEF0B774DF8864B, mem on close (mb), free: 201, total: 340, max: 672
2016-05-18 00:39:08,211 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D7B5BA69DB954A94BAD1677D15B3A6E5: Starting flush of map output
2016-05-18 00:39:08,211 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D7B5BA69DB954A94BAD1677D15B3A6E5: Span0.length = 37, perItem = 24
2016-05-18 00:39:08,212 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D7B5BA69DB954A94BAD1677D15B3A6E5: done sorting span=0, length=37, time=1
2016-05-18 00:39:08,212 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D7B5BA69DB954A94BAD1677D15B3A6E5: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:08,213 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D7B5BA69DB954A94BAD1677D15B3A6E5: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-local-dir/output/attempt_1463531947963_0001_1_00_000000_0_10070_0/file.out
2016-05-18 00:39:08,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:08 Completed running task attempt: attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,218 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:08,218 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531947963_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:08,219 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531947963_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531948205, Process_Duration=5, Process_End_Time=1463531948210, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:08,220 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531947963_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,220 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, taskAttemptId=attempt_1463531947963_0001_1_00_000000_0, creationTime=1463531948152, allocationTime=1463531948154, startTime=1463531948156, finishTime=1463531948220, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:08,220 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:08,221 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,221 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947963_0001_00_000001
2016-05-18 00:39:08,221 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531947963_0001_1_00_000000_0
2016-05-18 00:39:08,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_FINISHED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, taskId=task_1463531947963_0001_1_00_000000, startTime=1463531948156, finishTime=1463531948221, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463531947963_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531948205, Process_Duration=5, Process_End_Time=1463531948210, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:08,222 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:08,222 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_FINISHED]: vertexName=E4B70763FEA041B0AAEF0B774DF8864B, vertexId=vertex_1463531947963_0001_1_00, initRequestedTime=1463531948131, initedTime=1463531948147, startRequestedTime=1463531948132, startedTime=1463531948147, finishTime=1463531948222, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531948205, Process_Duration=5, Process_End_Time=1463531948210, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531948156, firstTasksToStart=[ task_1463531947963_0001_1_00_000000 ], lastTaskFinishTime=1463531948220, lastTasksToFinish=[ task_1463531947963_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531947963_0001_1_00_000000 ], longestDurationTasks=[ task_1463531947963_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:08,224 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:08,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531947963_0001_00_000001, stoppedTime=1463531948224, exitStatus=0
2016-05-18 00:39:08,224 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] attempt: attempt_1463531947963_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:08,225 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D7B5BA69DB954A94BAD1677D15B3A6E5
2016-05-18 00:39:08,225 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E4B70763FEA041B0AAEF0B774DF8864B, destinationVertexName=D7B5BA69DB954A94BAD1677D15B3A6E5. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:08,225 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,226 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:08,226 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,226 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531947963_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:08,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_STARTED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, taskId=task_1463531947963_0001_1_01_000000, scheduledTime=1463531948227, launchTime=1463531948227
2016-05-18 00:39:08,227 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531947963_0001_00_000002, launchTime=1463531948229
2016-05-18 00:39:08,230 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947963_0001_00_000002
2016-05-18 00:39:08,230 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,230 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531947963_0001_1_01_000000_0] started. Is using containerId: [container_1463531947963_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:08 Starting to run new task attempt: attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,231 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0, startTime=1463531948230, containerId=container_1463531947963_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:08,231 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,231 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531947963_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:08,232 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947963_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:08,232 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531947963_0001_00_000001, Exiting
2016-05-18 00:39:08,232 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531947963_0001_00_000001
2016-05-18 00:39:08,232 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531947963_0001_00_000001
2016-05-18 00:39:08,232 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:08,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: D7B5BA69DB954A94BAD1677D15B3A6E5, VertexParallelism: 1, TaskAttemptID:attempt_1463531947963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4B70763FEA041B0AAEF0B774DF8864B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A0664B4A2B374E6F9219EA4A751635FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:08,233 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:08,234 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531947963_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:08,234 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531947963_0001_00_000001 completed successfully
2016-05-18 00:39:08,236 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,237 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531947963_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,239 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531947963_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:08,239 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:08,242 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:08,247 INFO output.MROutput (MROutput.java:initialize(421)) - A0664B4A2B374E6F9219EA4A751635FE: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:08,247 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D7B5BA69DB954A94BAD1677D15B3A6E5, ordinal: 1
2016-05-18 00:39:08,248 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7B5BA69DB954A94BAD1677D15B3A6E5, mem on start (mb), free: 196, total: 340, max: 672
2016-05-18 00:39:08,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:08,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:08,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:08,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:08,249 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A0664B4A2B374E6F9219EA4A751635FE:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E4B70763FEA041B0AAEF0B774DF8864B:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:08,249 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:08,250 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:08,250 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:08,250 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A0664B4A2B374E6F9219EA4A751635FE:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E4B70763FEA041B0AAEF0B774DF8864B:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:08,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:08,251 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E4B70763FEA041B0AAEF0B774DF8864B: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:08,251 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E4B70763FEA041B0AAEF0B774DF8864B: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:08,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E4B70763FEA041B0AAEF0B774DF8864B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:08,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E4B70763FEA041B0AAEF0B774DF8864B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:08,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:08,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:08,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E4B70763FEA041B0AAEF0B774DF8864B
2016-05-18 00:39:08,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:08,254 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,255 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E4B70763FEA041B0AAEF0B774DF8864B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:08,257 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531947963_0001_1_00_000000_0_10070} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531948256, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:08,257 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E4B70763FEA041B0AAEF0B774DF8864B
2016-05-18 00:39:08,257 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:08,257 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:08,257 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E4B70763FEA041B0AAEF0B774DF8864B, wasInterrupted=false
2016-05-18 00:39:08,258 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:08,258 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:08,259 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001_wd/localmode-local-dir/output/attempt_1463531947963_0001_1_00_000000_0_10070/file.out, len=732, isLocal=true
2016-05-18 00:39:08,259 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:08,259 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:08,259 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:08,259 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:08,259 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E4B70763FEA041B0AAEF0B774DF8864B
2016-05-18 00:39:08,259 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - E4B70763FEA041B0AAEF0B774DF8864B: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:08,259 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 587265A775424EABB9DC2E4D45B25F76
2016-05-18 00:39:08,260 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"], id: A0664B4A2B374E6F9219EA4A751635FE
2016-05-18 00:39:08,260 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"]
2016-05-18 00:39:08,261 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 587265A775424EABB9DC2E4D45B25F76, for 1 inputs
2016-05-18 00:39:08,261 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D7B5BA69DB954A94BAD1677D15B3A6E5, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:08,259 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E4B70763FEA041B0AAEF0B774DF8864B: Shuffle Runner thread complete
2016-05-18 00:39:08,262 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E4B70763FEA041B0AAEF0B774DF8864B: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:08,259 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:08,263 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:08,269 INFO output.MROutput (MROutput.java:close(526)) - A0664B4A2B374E6F9219EA4A751635FE closed
2016-05-18 00:39:08,269 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531947963_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:08,270 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,270 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7B5BA69DB954A94BAD1677D15B3A6E5, mem on close (mb), free: 193, total: 340, max: 672
2016-05-18 00:39:08,270 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E4B70763FEA041B0AAEF0B774DF8864B
2016-05-18 00:39:08,271 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E4B70763FEA041B0AAEF0B774DF8864B: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:08,271 INFO output.MROutput (MROutput.java:close(526)) - A0664B4A2B374E6F9219EA4A751635FE closed
2016-05-18 00:39:08,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:08 Completed running task attempt: attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,272 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:08,272 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531947963_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:08,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531947963_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531948261, Process_Duration=9, Process_End_Time=1463531948270, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:08,275 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531947963_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, taskAttemptId=attempt_1463531947963_0001_1_01_000000_0, creationTime=1463531948227, allocationTime=1463531948228, startTime=1463531948230, finishTime=1463531948275, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:08,275 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,276 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:08,276 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531947963_0001_1_01_000000_0
2016-05-18 00:39:08,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:TASK_FINISHED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, taskId=task_1463531947963_0001_1_01_000000, startTime=1463531948230, finishTime=1463531948276, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463531947963_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531948261, Process_Duration=9, Process_End_Time=1463531948270, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:08,277 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:08,277 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5]
2016-05-18 00:39:08,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:VERTEX_FINISHED]: vertexName=D7B5BA69DB954A94BAD1677D15B3A6E5, vertexId=vertex_1463531947963_0001_1_01, initRequestedTime=1463531948132, initedTime=1463531948139, startRequestedTime=1463531948148, startedTime=1463531948148, finishTime=1463531948277, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531948261, Process_Duration=9, Process_End_Time=1463531948270, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531948230, firstTasksToStart=[ task_1463531947963_0001_1_01_000000 ], lastTaskFinishTime=1463531948275, lastTasksToFinish=[ task_1463531947963_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531947963_0001_1_01_000000 ], longestDurationTasks=[ task_1463531947963_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:08,278 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:08,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531947963_0001_00_000002, stoppedTime=1463531948278, exitStatus=0
2016-05-18 00:39:08,279 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:08,279 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,279 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531947963_0001_1_00 [E4B70763FEA041B0AAEF0B774DF8864B]
2016-05-18 00:39:08,279 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531947963_0001_1
2016-05-18 00:39:08,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531947963_0001_1
2016-05-18 00:39:08,279 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947963_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:08,281 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531947963_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:08,281 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A0664B4A2B374E6F9219EA4A751635FE for vertex: vertex_1463531947963_0001_1_01 [D7B5BA69DB954A94BAD1677D15B3A6E5], outputName: A0664B4A2B374E6F9219EA4A751635FE
2016-05-18 00:39:08,281 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,281 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531947963_0001_00_000002
2016-05-18 00:39:08,282 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531947963_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:08,282 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531947963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:08,282 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531947963_0001_00_000002, Exiting
2016-05-18 00:39:08,283 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531947963_0001_00_000002
2016-05-18 00:39:08,283 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531947963_0001_00_000002
2016-05-18 00:39:08,283 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531947963_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:08,283 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531947963_0001_00_000002 completed successfully
2016-05-18 00:39:08,289 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A0664B4A2B374E6F9219EA4A751635FE of vertex/vertexGroup:D7B5BA69DB954A94BAD1677D15B3A6E5 isVertexGroupOutput:false
2016-05-18 00:39:08,289 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,291 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531947963_0001_1, queueSize=0
2016-05-18 00:39:08,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531947963_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531947963_0001_1, startTime=1463531948130, finishTime=1463531948289, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=290, AM_GC_TIME_MILLIS=8, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063896466, Process_Duration=14, Process_End_Time=2927063896480, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:08,292 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531947963_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:08,292 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531947963_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:08 Completed Dag: dag_1463531947963_0001_1
2016-05-18 00:39:08,292 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531947963_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:08,292 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531947963_0001_1_post
2016-05-18 00:39:08,293 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,293 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:08,293 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:08,293 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false, with id=dag_1463531947963_0001_1
2016-05-18 00:39:08,293 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68] comprehensivesort/(1/1) ...ve/false_false_true_false, with id=dag_1463531947963_0001_1
2016-05-18 00:39:08,305 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1463531947963_0001
2016-05-18 00:39:08,309 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:08,309 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:08,309 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:08,309 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:08,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:08,310 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:08,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:08,311 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:08,311 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:08,311 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:08,312 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:08,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:08,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:08,313 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:08,316 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/86AAADC6F2F44721A17BF6FA9BBBB0EA/D99EFBDD3F9F46F1A3E1F876E8520E68/.tez/application_1463531947963_0001
2016-05-18 00:39:08,317 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false/_temporary
2016-05-18 00:39:08,317 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.377
2016-05-18 00:39:08,392 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:08,393 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:08,441 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 00:39:08,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-18 00:39:08,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:08,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:08,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:08,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:08,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false
2016-05-18 00:39:08,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:08,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:08,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"]
2016-05-18 00:39:08,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:08,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:08,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:08,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_true_false
2016-05-18 00:39:08,526 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:08,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:08,543 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:08,543 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:08,543 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:08,552 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001 doesn't exist and is created
2016-05-18 00:39:08,567 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:08,567 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:08,568 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd
2016-05-18 00:39:08,581 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531948543_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:08,581 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:08,581 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:08,583 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:08,583 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:08,583 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:08,583 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:08,584 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:08,585 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:08,585 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:08,585 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:08,586 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:08,586 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:08,586 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:08,586 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:08,586 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:08,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:08,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:08,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:08,588 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:08,589 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:08,590 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:08,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:08,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-log-dir
2016-05-18 00:39:08,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-log-dir/history.txt.appattempt_1463531948543_0001_000000, maxErrors=10
2016-05-18 00:39:08,591 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:08,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531948543_0001_000000, appSubmitTime=1463531948580, launchTime=1463531948580
2016-05-18 00:39:08,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:08,602 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34952
2016-05-18 00:39:08,602 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531948543
2016-05-18 00:39:08,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531948543_0001_000000, startTime=1463531948603
2016-05-18 00:39:08,603 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:08,668 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:08,668 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:08,668 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1463531948543_0001, dagName=[F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-05-18 00:39:08,686 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-05-18 00:39:08,686 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:08,687 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531948543_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-log-dir/dag_1463531948543_0001_1.dot
2016-05-18 00:39:08,687 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-log-dir/dag_1463531948543_0001_1-tez-dag.pb.txt
2016-05-18 00:39:08,693 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531948543_0001_1
2016-05-18 00:39:08,693 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,693 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-05-18 00:39:08 Running Dag: dag_1463531948543_0001_1
2016-05-18 00:39:08,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531948543_0001_1, submitTime=1463531948686
2016-05-18 00:39:08,703 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001/tez-conf.pb" } size: 36903 timestamp: 1463531948000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:08,703 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,703 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:08,704 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001/tez-conf.pb" } size: 36903 timestamp: 1463531948000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:08,704 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,704 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:08,704 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:08,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531948543_0001_1, initTime=1463531948704
2016-05-18 00:39:08,705 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531948543_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:08,705 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:08,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:DAG_STARTED]: dagID=dag_1463531948543_0001_1, startTime=1463531948705
2016-05-18 00:39:08,705 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531948543_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: AE6F37CA1A8B4581867301C7B97BC6C6 : {70D1B99369CA4DD284B66C1EFD884C29={InputName=70D1B99369CA4DD284B66C1EFD884C29}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 70D1B99369CA4DD284B66C1EFD884C29, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,706 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,707 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:08,707 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 70D1B99369CA4DD284B66C1EFD884C29 on vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,707 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,710 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1463531948543_0001, dagName=[F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-05-18 00:39:08,711 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 70D1B99369CA4DD284B66C1EFD884C29 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:08,714 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:08,714 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:08,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,714 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:08,715 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:08,715 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:08,715 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531948543_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:08,716 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957], numAdditionalOutputs=1
2016-05-18 00:39:08,716 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=5724FE37CFAA46A7A04F93ACCFDF434B, vertex=vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:08,716 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 70D1B99369CA4DD284B66C1EFD884C29 on vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,719 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EC12B85A0CB2417287791EB6B1D38957:5724FE37CFAA46A7A04F93ACCFDF434B using oldmapred API
2016-05-18 00:39:08,719 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=5724FE37CFAA46A7A04F93ACCFDF434B, vertexName=EC12B85A0CB2417287791EB6B1D38957, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:08,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, vertexId=vertex_1463531948543_0001_1_01, initRequestedTime=1463531948707, initedTime=1463531948716, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:08,721 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:08,721 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:08,722 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {70D1B99369CA4DD284B66C1EFD884C29=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:08,723 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] parallelism set to 1
2016-05-18 00:39:08,723 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:08,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, vertexId=vertex_1463531948543_0001_1_00, initRequestedTime=1463531948706, initedTime=1463531948723, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:08,723 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] with distanceFromRoot: 0
2016-05-18 00:39:08,724 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:08,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531948543_0001_1_00, startRequestedTime=1463531948707, startedTime=1463531948724
2016-05-18 00:39:08,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531948543_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:08,725 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:08,726 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531948543_0001_1_00 for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] numStartedSources: 1 numSources: 1
2016-05-18 00:39:08,726 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] with distanceFromRoot: 1
2016-05-18 00:39:08,726 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in AE6F37CA1A8B4581867301C7B97BC6C6
2016-05-18 00:39:08,726 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531948543_0001_1_01, startRequestedTime=1463531948725, startedTime=1463531948726
2016-05-18 00:39:08,729 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:08,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_STARTED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, taskId=task_1463531948543_0001_1_00_000000, scheduledTime=1463531948730, launchTime=1463531948730
2016-05-18 00:39:08,730 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,730 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: EC12B85A0CB2417287791EB6B1D38957 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:08,732 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: AE6F37CA1A8B4581867301C7B97BC6C6 in vertex: EC12B85A0CB2417287791EB6B1D38957 numBipartiteSourceTasks: 1
2016-05-18 00:39:08,733 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:08,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531948543_0001_00_000001, launchTime=1463531948734
2016-05-18 00:39:08,735 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531948543_0001_00_000001
2016-05-18 00:39:08,735 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531948543_0001_1_00_000000_0] started. Is using containerId: [container_1463531948543_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:08,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0, startTime=1463531948735, containerId=container_1463531948543_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:08,736 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531948543_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531948543_0001, with dag id: dag_1463531948543_0001_1
2016-05-18 00:39:08 Starting to run new task attempt: attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,737 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,737 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,737 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:08,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: AE6F37CA1A8B4581867301C7B97BC6C6, VertexParallelism: 1, TaskAttemptID:attempt_1463531948543_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70D1B99369CA4DD284B66C1EFD884C29, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EC12B85A0CB2417287791EB6B1D38957, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:08,738 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:08,739 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,741 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531948543_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:08,743 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531948543_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:08,745 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:08,745 INFO input.MRInput (MRInput.java:initialize(440)) - 70D1B99369CA4DD284B66C1EFD884C29 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:08,746 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 70D1B99369CA4DD284B66C1EFD884C29
2016-05-18 00:39:08,751 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE6F37CA1A8B4581867301C7B97BC6C6, ordinal: 0
2016-05-18 00:39:08,751 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE6F37CA1A8B4581867301C7B97BC6C6, mem on start (mb), free: 98, total: 340, max: 672
2016-05-18 00:39:08,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:08,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:08,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:08,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:08,752 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[70D1B99369CA4DD284B66C1EFD884C29:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EC12B85A0CB2417287791EB6B1D38957:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:08,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:08,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:08,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:08,753 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[70D1B99369CA4DD284B66C1EFD884C29:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EC12B85A0CB2417287791EB6B1D38957:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:08,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:08,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 70D1B99369CA4DD284B66C1EFD884C29 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:08,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:08,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:08,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 70D1B99369CA4DD284B66C1EFD884C29
2016-05-18 00:39:08,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:08,754 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,757 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:08,758 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 70D1B99369CA4DD284B66C1EFD884C29
2016-05-18 00:39:08,758 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: B1CC8BA530874850ACB37D400700DAC5
2016-05-18 00:39:08,758 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B1CC8BA530874850ACB37D400700DAC5
2016-05-18 00:39:08,759 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:08,760 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 70D1B99369CA4DD284B66C1EFD884C29 initialized RecordReader from event
2016-05-18 00:39:08,761 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - EC12B85A0CB2417287791EB6B1D38957 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@2427a9e, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@10bd5241, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:08,761 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for EC12B85A0CB2417287791EB6B1D38957: , UsingHashComparator=false
2016-05-18 00:39:08,782 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:08,783 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:08,783 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for EC12B85A0CB2417287791EB6B1D38957: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:08,785 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - EC12B85A0CB2417287791EB6B1D38957: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:08,785 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:08,787 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE6F37CA1A8B4581867301C7B97BC6C6, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:08,794 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,795 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE6F37CA1A8B4581867301C7B97BC6C6, mem on close (mb), free: 200, total: 341, max: 672
2016-05-18 00:39:08,795 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - EC12B85A0CB2417287791EB6B1D38957: Starting flush of map output
2016-05-18 00:39:08,796 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - EC12B85A0CB2417287791EB6B1D38957: Span0.length = 37, perItem = 24
2016-05-18 00:39:08,798 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - EC12B85A0CB2417287791EB6B1D38957: done sorting span=0, length=37, time=2
2016-05-18 00:39:08,799 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - EC12B85A0CB2417287791EB6B1D38957: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:08,801 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - EC12B85A0CB2417287791EB6B1D38957: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-local-dir/output/attempt_1463531948543_0001_1_00_000000_0_10077_0/file.out
2016-05-18 00:39:08,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:08 Completed running task attempt: attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,806 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:08,807 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531948543_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:08,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531948543_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531948787, Process_Duration=7, Process_End_Time=1463531948794, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:08,811 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531948543_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, taskAttemptId=attempt_1463531948543_0001_1_00_000000_0, creationTime=1463531948730, allocationTime=1463531948733, startTime=1463531948735, finishTime=1463531948811, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:08,811 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531948543_0001_1_00_000000_0
2016-05-18 00:39:08,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_FINISHED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, taskId=task_1463531948543_0001_1_00_000000, startTime=1463531948735, finishTime=1463531948812, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1463531948543_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531948787, Process_Duration=7, Process_End_Time=1463531948794, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:08,811 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,812 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:08,813 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,812 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:08,814 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_FINISHED]: vertexName=AE6F37CA1A8B4581867301C7B97BC6C6, vertexId=vertex_1463531948543_0001_1_00, initRequestedTime=1463531948706, initedTime=1463531948723, startRequestedTime=1463531948707, startedTime=1463531948724, finishTime=1463531948814, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531948787, Process_Duration=7, Process_End_Time=1463531948794, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531948735, firstTasksToStart=[ task_1463531948543_0001_1_00_000000 ], lastTaskFinishTime=1463531948811, lastTasksToFinish=[ task_1463531948543_0001_1_00_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531948543_0001_1_00_000000 ], longestDurationTasks=[ task_1463531948543_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:08,815 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:08,815 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] attempt: attempt_1463531948543_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:08,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531948543_0001_00_000001, stoppedTime=1463531948815, exitStatus=0
2016-05-18 00:39:08,815 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EC12B85A0CB2417287791EB6B1D38957
2016-05-18 00:39:08,816 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:08,816 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=AE6F37CA1A8B4581867301C7B97BC6C6, destinationVertexName=EC12B85A0CB2417287791EB6B1D38957. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:08,816 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,817 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,818 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531948543_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:08,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_STARTED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, taskId=task_1463531948543_0001_1_01_000000, scheduledTime=1463531948818, launchTime=1463531948818
2016-05-18 00:39:08,818 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531948543_0001_00_000001
2016-05-18 00:39:08,818 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,818 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531948543_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:08,818 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531948543_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:08,818 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531948543_0001_00_000001, Exiting
2016-05-18 00:39:08,819 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531948543_0001_00_000001
2016-05-18 00:39:08,819 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531948543_0001_00_000001
2016-05-18 00:39:08,823 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531948543_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:08,823 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531948543_0001_00_000001 completed successfully
2016-05-18 00:39:08,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531948543_0001_00_000002, launchTime=1463531948824
2016-05-18 00:39:08,824 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531948543_0001_00_000002
2016-05-18 00:39:08,825 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531948543_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08 Starting to run new task attempt: attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531948543_0001_1_01_000000_0] started. Is using containerId: [container_1463531948543_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:08,826 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,827 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,827 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:08,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: EC12B85A0CB2417287791EB6B1D38957, VertexParallelism: 1, TaskAttemptID:attempt_1463531948543_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE6F37CA1A8B4581867301C7B97BC6C6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5724FE37CFAA46A7A04F93ACCFDF434B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:08,827 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:08,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0, startTime=1463531948825, containerId=container_1463531948543_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:08,827 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,830 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:08,831 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531948543_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,831 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531948543_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:08,833 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:08,841 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EC12B85A0CB2417287791EB6B1D38957, ordinal: 1
2016-05-18 00:39:08,841 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC12B85A0CB2417287791EB6B1D38957, mem on start (mb), free: 195, total: 341, max: 672
2016-05-18 00:39:08,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:08,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:08,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:08,842 INFO output.MROutput (MROutput.java:initialize(421)) - 5724FE37CFAA46A7A04F93ACCFDF434B: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:08,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:08,842 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[AE6F37CA1A8B4581867301C7B97BC6C6:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [5724FE37CFAA46A7A04F93ACCFDF434B:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:08,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:08,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:08,843 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:08,843 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[AE6F37CA1A8B4581867301C7B97BC6C6:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [5724FE37CFAA46A7A04F93ACCFDF434B:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:08,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:08,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: AE6F37CA1A8B4581867301C7B97BC6C6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:08,844 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - AE6F37CA1A8B4581867301C7B97BC6C6: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:08,844 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - AE6F37CA1A8B4581867301C7B97BC6C6: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:08,845 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: AE6F37CA1A8B4581867301C7B97BC6C6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:08,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:08,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: AE6F37CA1A8B4581867301C7B97BC6C6
2016-05-18 00:39:08,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:08,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:08,850 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,851 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AE6F37CA1A8B4581867301C7B97BC6C6: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:08,853 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531948543_0001_1_00_000000_0_10077} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531948852, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:08,853 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : AE6F37CA1A8B4581867301C7B97BC6C6
2016-05-18 00:39:08,853 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:08,853 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: AE6F37CA1A8B4581867301C7B97BC6C6, wasInterrupted=false
2016-05-18 00:39:08,855 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:08,856 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:08,856 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:08,856 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: B1CC8BA530874850ACB37D400700DAC5
2016-05-18 00:39:08,857 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"], id: 5724FE37CFAA46A7A04F93ACCFDF434B
2016-05-18 00:39:08,857 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"]
2016-05-18 00:39:08,857 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B1CC8BA530874850ACB37D400700DAC5, for 1 inputs
2016-05-18 00:39:08,858 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001_wd/localmode-local-dir/output/attempt_1463531948543_0001_1_00_000000_0_10077/file.out, len=732, isLocal=true
2016-05-18 00:39:08,858 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:08,858 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:08,858 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:08,859 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:08,859 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : AE6F37CA1A8B4581867301C7B97BC6C6
2016-05-18 00:39:08,859 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EC12B85A0CB2417287791EB6B1D38957, all 1 inputs ready in: 00:00:00.002
2016-05-18 00:39:08,859 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - AE6F37CA1A8B4581867301C7B97BC6C6: Shuffle Runner thread complete
2016-05-18 00:39:08,860 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - AE6F37CA1A8B4581867301C7B97BC6C6: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:08,866 INFO output.MROutput (MROutput.java:close(526)) - 5724FE37CFAA46A7A04F93ACCFDF434B closed
2016-05-18 00:39:08,867 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531948543_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:08,867 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,868 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EC12B85A0CB2417287791EB6B1D38957, mem on close (mb), free: 192, total: 341, max: 672
2016-05-18 00:39:08,868 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: AE6F37CA1A8B4581867301C7B97BC6C6
2016-05-18 00:39:08,868 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - AE6F37CA1A8B4581867301C7B97BC6C6: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:08,869 INFO output.MROutput (MROutput.java:close(526)) - 5724FE37CFAA46A7A04F93ACCFDF434B closed
2016-05-18 00:39:08,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:08 Completed running task attempt: attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,869 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:08,869 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531948543_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:08,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531948543_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531948859, Process_Duration=8, Process_End_Time=1463531948867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:08,871 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531948543_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, taskAttemptId=attempt_1463531948543_0001_1_01_000000_0, creationTime=1463531948818, allocationTime=1463531948819, startTime=1463531948825, finishTime=1463531948871, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:08,871 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:08,872 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:08,872 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,872 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531948543_0001_1_01_000000_0
2016-05-18 00:39:08,872 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531948543_0001_00_000002
2016-05-18 00:39:08,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:TASK_FINISHED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, taskId=task_1463531948543_0001_1_01_000000, startTime=1463531948825, finishTime=1463531948872, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463531948543_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531948859, Process_Duration=8, Process_End_Time=1463531948867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:08,873 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:08,873 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957]
2016-05-18 00:39:08,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:VERTEX_FINISHED]: vertexName=EC12B85A0CB2417287791EB6B1D38957, vertexId=vertex_1463531948543_0001_1_01, initRequestedTime=1463531948707, initedTime=1463531948716, startRequestedTime=1463531948725, startedTime=1463531948726, finishTime=1463531948874, timeTaken=148, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531948859, Process_Duration=8, Process_End_Time=1463531948867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531948825, firstTasksToStart=[ task_1463531948543_0001_1_01_000000 ], lastTaskFinishTime=1463531948871, lastTasksToFinish=[ task_1463531948543_0001_1_01_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531948543_0001_1_01_000000 ], longestDurationTasks=[ task_1463531948543_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:08,874 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:08,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531948543_0001_00_000002, stoppedTime=1463531948874, exitStatus=0
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531948543_0001_1_00 [AE6F37CA1A8B4581867301C7B97BC6C6]
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531948543_0001_1
2016-05-18 00:39:08,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531948543_0001_1
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531948543_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:08,875 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 5724FE37CFAA46A7A04F93ACCFDF434B for vertex: vertex_1463531948543_0001_1_01 [EC12B85A0CB2417287791EB6B1D38957], outputName: 5724FE37CFAA46A7A04F93ACCFDF434B
2016-05-18 00:39:08,876 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531948543_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:08,883 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531948543_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:08,883 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531948543_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:08,883 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531948543_0001_00_000002, Exiting
2016-05-18 00:39:08,883 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531948543_0001_00_000002
2016-05-18 00:39:08,883 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:5724FE37CFAA46A7A04F93ACCFDF434B of vertex/vertexGroup:EC12B85A0CB2417287791EB6B1D38957 isVertexGroupOutput:false
2016-05-18 00:39:08,883 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531948543_0001_00_000002
2016-05-18 00:39:08,883 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:08,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531948543_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:08,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531948543_0001_00_000002 completed successfully
2016-05-18 00:39:08,886 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531948543_0001_1, queueSize=0
2016-05-18 00:39:08,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531948543_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531948543_0001_1, startTime=1463531948705, finishTime=1463531948884, timeTaken=179, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=360, AM_GC_TIME_MILLIS=7, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063897646, Process_Duration=15, Process_End_Time=2927063897661, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:08,887 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531948543_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:08,887 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531948543_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:08 Completed Dag: dag_1463531948543_0001_1
2016-05-18 00:39:08,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531948543_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:08,887 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531948543_0001_1_post
2016-05-18 00:39:08,887 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:08,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:08,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:08,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false, with id=dag_1463531948543_0001_1
2016-05-18 00:39:08,888 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA] comprehensivesort/(1/1) ...ive/true_false_true_false, with id=dag_1463531948543_0001_1
2016-05-18 00:39:08,895 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1463531948543_0001
2016-05-18 00:39:08,897 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:08,897 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:08,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:08,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:08,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:08,898 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:08,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:08,899 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:08,899 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:08,900 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:08,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:08,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:08,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:08,902 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:08,903 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:08,903 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:08,906 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F068EDA55A2A4224928443D29DD23D00/C7CA9D01FE3147F5A06054DCD4381FAA/.tez/application_1463531948543_0001
2016-05-18 00:39:08,907 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false/_temporary
2016-05-18 00:39:08,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.383
2016-05-18 00:39:08,998 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:08,998 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:09,048 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:39:09,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-18 00:39:09,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:09,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:09,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:09,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:09,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false
2016-05-18 00:39:09,167 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:09,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:09,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"]
2016-05-18 00:39:09,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:09,168 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:09,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:09,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_true_false
2016-05-18 00:39:09,170 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:09,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:09,197 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:09,197 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:09,197 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:09,209 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001 doesn't exist and is created
2016-05-18 00:39:09,224 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:09,224 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:09,225 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd
2016-05-18 00:39:09,243 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531949197_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:09,244 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:09,244 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:09,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:09,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:09,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:09,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:09,247 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:09,249 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:09,250 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:09,250 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:09,250 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:09,250 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:09,251 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:09,251 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:09,251 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:09,251 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:09,251 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:09,252 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:09,252 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:09,254 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:09,255 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:09,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:09,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-log-dir
2016-05-18 00:39:09,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-log-dir/history.txt.appattempt_1463531949197_0001_000000, maxErrors=10
2016-05-18 00:39:09,256 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:09,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531949197_0001_000000, appSubmitTime=1463531949242, launchTime=1463531949243
2016-05-18 00:39:09,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:09,264 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44672
2016-05-18 00:39:09,264 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531949197
2016-05-18 00:39:09,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531949197_0001_000000, startTime=1463531949267
2016-05-18 00:39:09,267 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:09,324 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:09,324 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:09,325 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1463531949197_0001, dagName=[F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-05-18 00:39:09,342 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-05-18 00:39:09,343 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:09,343 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531949197_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-log-dir/dag_1463531949197_0001_1.dot
2016-05-18 00:39:09,344 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-log-dir/dag_1463531949197_0001_1-tez-dag.pb.txt
2016-05-18 00:39:09,350 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531949197_0001_1
2016-05-18 00:39:09,350 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,351 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-05-18 00:39:09 Running Dag: dag_1463531949197_0001_1
2016-05-18 00:39:09,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531949197_0001_1, submitTime=1463531949342
2016-05-18 00:39:09,361 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001/tez-conf.pb" } size: 36903 timestamp: 1463531949000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:09,361 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,361 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:09,362 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001/tez-conf.pb" } size: 36903 timestamp: 1463531949000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:09,362 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,362 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:09,362 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:09,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531949197_0001_1, initTime=1463531949363
2016-05-18 00:39:09,363 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949197_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:09,363 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:09,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:DAG_STARTED]: dagID=dag_1463531949197_0001_1, startTime=1463531949363
2016-05-18 00:39:09,363 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949197_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C6315D0E9CC84CD8A0A8F44852F1BCB6 : {FDD74C11B854466691B12FC61B079C3E={InputName=FDD74C11B854466691B12FC61B079C3E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FDD74C11B854466691B12FC61B079C3E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,364 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,365 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1463531949197_0001, dagName=[F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-05-18 00:39:09,365 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:09,365 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FDD74C11B854466691B12FC61B079C3E on vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,365 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,369 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:09,369 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,369 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,370 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FDD74C11B854466691B12FC61B079C3E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:09,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531949197_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:09,371 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC], numAdditionalOutputs=1
2016-05-18 00:39:09,371 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=BEC6C996D66943379167A3E116AA0085, vertex=vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:09,372 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:09,373 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:09,373 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:09,373 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:09,374 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A79A9CAD3E794A5A979FFCDD8D5206BC:BEC6C996D66943379167A3E116AA0085 using oldmapred API
2016-05-18 00:39:09,374 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=BEC6C996D66943379167A3E116AA0085, vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:09,374 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FDD74C11B854466691B12FC61B079C3E on vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, vertexId=vertex_1463531949197_0001_1_01, initRequestedTime=1463531949365, initedTime=1463531949371, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:09,375 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:09,376 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:09,377 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FDD74C11B854466691B12FC61B079C3E=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:09,378 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] parallelism set to 1
2016-05-18 00:39:09,378 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:09,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, vertexId=vertex_1463531949197_0001_1_00, initRequestedTime=1463531949364, initedTime=1463531949378, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:09,378 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] with distanceFromRoot: 0
2016-05-18 00:39:09,383 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:09,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531949197_0001_1_00, startRequestedTime=1463531949365, startedTime=1463531949384
2016-05-18 00:39:09,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531949197_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:09,386 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:09,386 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531949197_0001_1_00 for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] numStartedSources: 1 numSources: 1
2016-05-18 00:39:09,386 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C6315D0E9CC84CD8A0A8F44852F1BCB6
2016-05-18 00:39:09,386 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,386 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] with distanceFromRoot: 1
2016-05-18 00:39:09,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531949197_0001_1_01, startRequestedTime=1463531949386, startedTime=1463531949388
2016-05-18 00:39:09,388 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:09,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_STARTED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, taskId=task_1463531949197_0001_1_00_000000, scheduledTime=1463531949388, launchTime=1463531949388
2016-05-18 00:39:09,389 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,392 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A79A9CAD3E794A5A979FFCDD8D5206BC with 0 source tasks and 1 pending tasks
2016-05-18 00:39:09,392 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:09,394 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: C6315D0E9CC84CD8A0A8F44852F1BCB6 in vertex: A79A9CAD3E794A5A979FFCDD8D5206BC numBipartiteSourceTasks: 1
2016-05-18 00:39:09,395 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531949197_0001, with dag id: dag_1463531949197_0001_1
2016-05-18 00:39:09,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531949197_0001_00_000001, launchTime=1463531949395
2016-05-18 00:39:09,401 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949197_0001_00_000001
2016-05-18 00:39:09,403 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531949197_0001_1_00_000000_0] started. Is using containerId: [container_1463531949197_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:09,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0, startTime=1463531949403, containerId=container_1463531949197_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:09,403 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949197_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09 Starting to run new task attempt: attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,404 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,404 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,404 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:09,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: C6315D0E9CC84CD8A0A8F44852F1BCB6, VertexParallelism: 1, TaskAttemptID:attempt_1463531949197_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDD74C11B854466691B12FC61B079C3E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:09,405 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:09,406 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531949197_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:09,406 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531949197_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:09,407 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,410 INFO input.MRInput (MRInput.java:initialize(440)) - FDD74C11B854466691B12FC61B079C3E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:09,410 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FDD74C11B854466691B12FC61B079C3E
2016-05-18 00:39:09,411 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:09,417 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C6315D0E9CC84CD8A0A8F44852F1BCB6, ordinal: 0
2016-05-18 00:39:09,417 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C6315D0E9CC84CD8A0A8F44852F1BCB6, mem on start (mb), free: 92, total: 341, max: 672
2016-05-18 00:39:09,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:09,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:09,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:09,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:09,418 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FDD74C11B854466691B12FC61B079C3E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A79A9CAD3E794A5A979FFCDD8D5206BC:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:09,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:09,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:09,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:09,419 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FDD74C11B854466691B12FC61B079C3E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A79A9CAD3E794A5A979FFCDD8D5206BC:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:09,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:09,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FDD74C11B854466691B12FC61B079C3E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:09,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:09,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:09,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FDD74C11B854466691B12FC61B079C3E
2016-05-18 00:39:09,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:09,420 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,422 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:09,422 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FDD74C11B854466691B12FC61B079C3E initialized RecordReader from event
2016-05-18 00:39:09,423 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:09,423 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: FDD74C11B854466691B12FC61B079C3E
2016-05-18 00:39:09,424 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: B9427D2DB876499CAD6478526201D875
2016-05-18 00:39:09,424 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B9427D2DB876499CAD6478526201D875
2016-05-18 00:39:09,427 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A79A9CAD3E794A5A979FFCDD8D5206BC using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@35a303a5, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@5045f606, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:09,428 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A79A9CAD3E794A5A979FFCDD8D5206BC: , UsingHashComparator=false
2016-05-18 00:39:09,449 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:09,450 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:09,450 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A79A9CAD3E794A5A979FFCDD8D5206BC: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:09,450 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A79A9CAD3E794A5A979FFCDD8D5206BC: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:09,451 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:09,451 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C6315D0E9CC84CD8A0A8F44852F1BCB6, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:09,456 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,456 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C6315D0E9CC84CD8A0A8F44852F1BCB6, mem on close (mb), free: 199, total: 341, max: 672
2016-05-18 00:39:09,456 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A79A9CAD3E794A5A979FFCDD8D5206BC: Starting flush of map output
2016-05-18 00:39:09,456 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A79A9CAD3E794A5A979FFCDD8D5206BC: Span0.length = 37, perItem = 24
2016-05-18 00:39:09,458 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A79A9CAD3E794A5A979FFCDD8D5206BC: done sorting span=0, length=37, time=1
2016-05-18 00:39:09,459 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A79A9CAD3E794A5A979FFCDD8D5206BC: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:09,460 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A79A9CAD3E794A5A979FFCDD8D5206BC: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-local-dir/output/attempt_1463531949197_0001_1_00_000000_0_10083_0/file.out
2016-05-18 00:39:09,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:09 Completed running task attempt: attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,464 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:09,464 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531949197_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:09,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531949197_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531949451, Process_Duration=5, Process_End_Time=1463531949456, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:09,468 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531949197_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:09,468 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:09,468 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:09,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, taskAttemptId=attempt_1463531949197_0001_1_00_000000_0, creationTime=1463531949389, allocationTime=1463531949394, startTime=1463531949403, finishTime=1463531949468, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:09,468 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,468 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949197_0001_00_000001
2016-05-18 00:39:09,468 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531949197_0001_1_00_000000_0
2016-05-18 00:39:09,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_FINISHED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, taskId=task_1463531949197_0001_1_00_000000, startTime=1463531949403, finishTime=1463531949469, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463531949197_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531949451, Process_Duration=5, Process_End_Time=1463531949456, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:09,469 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:09,470 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,471 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_FINISHED]: vertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, vertexId=vertex_1463531949197_0001_1_00, initRequestedTime=1463531949364, initedTime=1463531949378, startRequestedTime=1463531949365, startedTime=1463531949384, finishTime=1463531949470, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531949451, Process_Duration=5, Process_End_Time=1463531949456, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531949403, firstTasksToStart=[ task_1463531949197_0001_1_00_000000 ], lastTaskFinishTime=1463531949468, lastTasksToFinish=[ task_1463531949197_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531949197_0001_1_00_000000 ], longestDurationTasks=[ task_1463531949197_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:09,473 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:09,473 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] attempt: attempt_1463531949197_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:09,473 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A79A9CAD3E794A5A979FFCDD8D5206BC
2016-05-18 00:39:09,473 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, destinationVertexName=A79A9CAD3E794A5A979FFCDD8D5206BC. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:09,473 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531949197_0001_00_000001, stoppedTime=1463531949474, exitStatus=0
2016-05-18 00:39:09,474 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:09,474 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:09,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_STARTED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, taskId=task_1463531949197_0001_1_01_000000, scheduledTime=1463531949474, launchTime=1463531949474
2016-05-18 00:39:09,475 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531949197_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:09,475 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531949197_0001_00_000002, launchTime=1463531949476
2016-05-18 00:39:09,477 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949197_0001_00_000002
2016-05-18 00:39:09,478 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949197_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,478 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531949197_0001_1_01_000000_0] started. Is using containerId: [container_1463531949197_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:09 Starting to run new task attempt: attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0, startTime=1463531949478, containerId=container_1463531949197_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:09,478 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,478 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,478 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:09,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: A79A9CAD3E794A5A979FFCDD8D5206BC, VertexParallelism: 1, TaskAttemptID:attempt_1463531949197_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C6315D0E9CC84CD8A0A8F44852F1BCB6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BEC6C996D66943379167A3E116AA0085, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:09,479 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531949197_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:09,479 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949197_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:09,479 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:09,479 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531949197_0001_00_000001, Exiting
2016-05-18 00:39:09,480 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531949197_0001_00_000001
2016-05-18 00:39:09,480 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531949197_0001_00_000001
2016-05-18 00:39:09,480 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,481 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531949197_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,482 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531949197_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:09,482 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531949197_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:09,483 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531949197_0001_00_000001 completed successfully
2016-05-18 00:39:09,485 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:09,486 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:09,491 INFO output.MROutput (MROutput.java:initialize(421)) - BEC6C996D66943379167A3E116AA0085: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:09,494 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A79A9CAD3E794A5A979FFCDD8D5206BC, ordinal: 1
2016-05-18 00:39:09,494 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A79A9CAD3E794A5A979FFCDD8D5206BC, mem on start (mb), free: 194, total: 341, max: 672
2016-05-18 00:39:09,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:09,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:09,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:09,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:09,495 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[BEC6C996D66943379167A3E116AA0085:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [C6315D0E9CC84CD8A0A8F44852F1BCB6:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:09,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:09,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:09,496 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:09,497 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[BEC6C996D66943379167A3E116AA0085:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [C6315D0E9CC84CD8A0A8F44852F1BCB6:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:09,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:09,498 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:09,498 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:09,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C6315D0E9CC84CD8A0A8F44852F1BCB6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:09,499 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: C6315D0E9CC84CD8A0A8F44852F1BCB6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:09,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:09,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:09,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C6315D0E9CC84CD8A0A8F44852F1BCB6
2016-05-18 00:39:09,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:09,503 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,503 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:09,505 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531949197_0001_1_00_000000_0_10083} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531949504, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 00:39:09,505 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : C6315D0E9CC84CD8A0A8F44852F1BCB6
2016-05-18 00:39:09,505 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:09,505 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: C6315D0E9CC84CD8A0A8F44852F1BCB6, wasInterrupted=false
2016-05-18 00:39:09,506 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:09,507 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:09,507 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:09,508 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: B9427D2DB876499CAD6478526201D875
2016-05-18 00:39:09,508 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"], id: BEC6C996D66943379167A3E116AA0085
2016-05-18 00:39:09,508 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001_wd/localmode-local-dir/output/attempt_1463531949197_0001_1_00_000000_0_10083/file.out, len=732, isLocal=true
2016-05-18 00:39:09,509 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:09,509 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:09,509 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:09,509 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"]
2016-05-18 00:39:09,510 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B9427D2DB876499CAD6478526201D875, for 1 inputs
2016-05-18 00:39:09,510 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:09,510 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : C6315D0E9CC84CD8A0A8F44852F1BCB6
2016-05-18 00:39:09,511 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: Shuffle Runner thread complete
2016-05-18 00:39:09,510 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A79A9CAD3E794A5A979FFCDD8D5206BC, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:09,513 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:09,518 INFO output.MROutput (MROutput.java:close(526)) - BEC6C996D66943379167A3E116AA0085 closed
2016-05-18 00:39:09,520 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531949197_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:09,521 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,521 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A79A9CAD3E794A5A979FFCDD8D5206BC, mem on close (mb), free: 191, total: 341, max: 672
2016-05-18 00:39:09,521 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: C6315D0E9CC84CD8A0A8F44852F1BCB6
2016-05-18 00:39:09,521 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C6315D0E9CC84CD8A0A8F44852F1BCB6: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:09,521 INFO output.MROutput (MROutput.java:close(526)) - BEC6C996D66943379167A3E116AA0085 closed
2016-05-18 00:39:09,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:09 Completed running task attempt: attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,522 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:09,522 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531949197_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:09,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531949197_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531949511, Process_Duration=10, Process_End_Time=1463531949521, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:09,526 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531949197_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:09,526 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:09,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:09,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, taskAttemptId=attempt_1463531949197_0001_1_01_000000_0, creationTime=1463531949475, allocationTime=1463531949476, startTime=1463531949478, finishTime=1463531949527, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:09,527 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531949197_0001_1_01_000000_0
2016-05-18 00:39:09,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:TASK_FINISHED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, taskId=task_1463531949197_0001_1_01_000000, startTime=1463531949478, finishTime=1463531949527, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463531949197_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531949511, Process_Duration=10, Process_End_Time=1463531949521, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:09,528 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:09,528 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC]
2016-05-18 00:39:09,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:VERTEX_FINISHED]: vertexName=A79A9CAD3E794A5A979FFCDD8D5206BC, vertexId=vertex_1463531949197_0001_1_01, initRequestedTime=1463531949365, initedTime=1463531949371, startRequestedTime=1463531949386, startedTime=1463531949388, finishTime=1463531949528, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531949511, Process_Duration=10, Process_End_Time=1463531949521, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531949478, firstTasksToStart=[ task_1463531949197_0001_1_01_000000 ], lastTaskFinishTime=1463531949527, lastTasksToFinish=[ task_1463531949197_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531949197_0001_1_01_000000 ], longestDurationTasks=[ task_1463531949197_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:09,530 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:09,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531949197_0001_00_000002, stoppedTime=1463531949530, exitStatus=0
2016-05-18 00:39:09,530 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:09,530 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:09,530 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531949197_0001_1_00 [C6315D0E9CC84CD8A0A8F44852F1BCB6]
2016-05-18 00:39:09,530 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531949197_0001_1
2016-05-18 00:39:09,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531949197_0001_1
2016-05-18 00:39:09,531 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949197_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:09,531 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: BEC6C996D66943379167A3E116AA0085 for vertex: vertex_1463531949197_0001_1_01 [A79A9CAD3E794A5A979FFCDD8D5206BC], outputName: BEC6C996D66943379167A3E116AA0085
2016-05-18 00:39:09,534 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949197_0001_00_000002
2016-05-18 00:39:09,534 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531949197_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:09,534 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949197_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:09,534 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531949197_0001_00_000002, Exiting
2016-05-18 00:39:09,534 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531949197_0001_00_000002
2016-05-18 00:39:09,535 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531949197_0001_00_000002
2016-05-18 00:39:09,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531949197_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:09,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531949197_0001_00_000002 completed successfully
2016-05-18 00:39:09,534 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531949197_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:09,540 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:BEC6C996D66943379167A3E116AA0085 of vertex/vertexGroup:A79A9CAD3E794A5A979FFCDD8D5206BC isVertexGroupOutput:false
2016-05-18 00:39:09,540 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:09,542 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531949197_0001_1, queueSize=0
2016-05-18 00:39:09,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949197_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531949197_0001_1, startTime=1463531949363, finishTime=1463531949540, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=320, AM_GC_TIME_MILLIS=7, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063898962, Process_Duration=15, Process_End_Time=2927063898977, Read_Duration=5, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:09,542 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531949197_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:09,542 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949197_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:09 Completed Dag: dag_1463531949197_0001_1
2016-05-18 00:39:09,543 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531949197_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:09,543 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531949197_0001_1_post
2016-05-18 00:39:09,543 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,543 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:09,543 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:09,543 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false, with id=dag_1463531949197_0001_1
2016-05-18 00:39:09,544 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F] comprehensivesort/(1/1) ...sive/true_true_true_false, with id=dag_1463531949197_0001_1
2016-05-18 00:39:09,552 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1463531949197_0001
2016-05-18 00:39:09,554 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:09,554 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:09,555 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:09,555 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:09,555 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:09,555 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:09,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:09,556 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:09,556 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:09,556 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:09,556 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:09,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:09,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:09,557 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:09,560 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/F45CF3E19B4047618BEA8DF0CB8765DF/BEE1830AD71941258CF8DF62E133D89F/.tez/application_1463531949197_0001
2016-05-18 00:39:09,561 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false/_temporary
2016-05-18 00:39:09,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.394
2016-05-18 00:39:09,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:09,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:09,633 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:09,633 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:09,670 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 00:39:09,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:39:09,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:09,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:09,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:09,698 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:09,704 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false
2016-05-18 00:39:09,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:09,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:09,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"]
2016-05-18 00:39:09,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:09,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:09,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:09,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_true_false
2016-05-18 00:39:09,755 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:09,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:09,776 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:09,777 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:09,778 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:09,789 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001 doesn't exist and is created
2016-05-18 00:39:09,805 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:09,805 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:09,806 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd
2016-05-18 00:39:09,816 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531949777_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:09,816 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:09,817 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:09,818 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:09,818 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:09,818 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:09,819 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:09,819 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:09,820 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:09,821 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:09,821 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:09,821 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:09,821 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:09,822 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:09,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:09,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:09,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:09,822 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:09,823 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:09,823 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:09,824 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:09,825 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:09,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:09,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-log-dir
2016-05-18 00:39:09,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-log-dir/history.txt.appattempt_1463531949777_0001_000000, maxErrors=10
2016-05-18 00:39:09,826 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:09,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531949777_0001_000000, appSubmitTime=1463531949816, launchTime=1463531949816
2016-05-18 00:39:09,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:09,831 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55779
2016-05-18 00:39:09,832 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531949777
2016-05-18 00:39:09,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531949777_0001_000000, startTime=1463531949836
2016-05-18 00:39:09,836 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:09,906 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:09,906 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:09,906 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1463531949777_0001, dagName=[FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-05-18 00:39:09,923 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-05-18 00:39:09,923 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:09,924 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531949777_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-log-dir/dag_1463531949777_0001_1.dot
2016-05-18 00:39:09,924 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-log-dir/dag_1463531949777_0001_1-tez-dag.pb.txt
2016-05-18 00:39:09,934 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531949777_0001_1
2016-05-18 00:39:09,935 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,935 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-05-18 00:39:09 Running Dag: dag_1463531949777_0001_1
2016-05-18 00:39:09,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531949777_0001_1, submitTime=1463531949923
2016-05-18 00:39:09,944 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001/tez-conf.pb" } size: 36903 timestamp: 1463531949000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:09,945 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,945 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:09,946 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001/tez-conf.pb" } size: 36903 timestamp: 1463531949000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:09,946 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:09,946 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:09,946 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:09,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531949777_0001_1, initTime=1463531949947
2016-05-18 00:39:09,947 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949777_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:09,947 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:09,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:DAG_STARTED]: dagID=dag_1463531949777_0001_1, startTime=1463531949948
2016-05-18 00:39:09,948 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949777_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:09,948 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: FD011450C4FA4A3A92E2266968E085EF : {BA3FA97452F84C97B1102E8DDCEF4AD9={InputName=BA3FA97452F84C97B1102E8DDCEF4AD9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:09,948 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: BA3FA97452F84C97B1102E8DDCEF4AD9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:09,948 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,949 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,949 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,950 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:09,950 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:09,950 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: BA3FA97452F84C97B1102E8DDCEF4AD9 on vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,952 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input BA3FA97452F84C97B1102E8DDCEF4AD9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:09,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:09,953 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1463531949777_0001, dagName=[FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-05-18 00:39:09,955 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:09,955 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:09,955 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:09,956 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:09,955 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:09,956 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: BA3FA97452F84C97B1102E8DDCEF4AD9 on vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,957 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:09,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531949777_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:09,959 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82], numAdditionalOutputs=1
2016-05-18 00:39:09,960 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=2F2E1BB218D24CA391E4CEEF432F0513, vertex=vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:09,962 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D80869F0FFA344B9A3F11AA006776E82:2F2E1BB218D24CA391E4CEEF432F0513 using oldmapred API
2016-05-18 00:39:09,962 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=2F2E1BB218D24CA391E4CEEF432F0513, vertexName=D80869F0FFA344B9A3F11AA006776E82, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:09,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, vertexId=vertex_1463531949777_0001_1_01, initRequestedTime=1463531949950, initedTime=1463531949959, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:09,963 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:09,965 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:09,965 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {BA3FA97452F84C97B1102E8DDCEF4AD9=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:09,965 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] parallelism set to 1
2016-05-18 00:39:09,966 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:09,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, vertexId=vertex_1463531949777_0001_1_00, initRequestedTime=1463531949948, initedTime=1463531949966, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:09,966 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] with distanceFromRoot: 0
2016-05-18 00:39:09,966 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:09,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531949777_0001_1_00, startRequestedTime=1463531949950, startedTime=1463531949966
2016-05-18 00:39:09,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531949777_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:09,967 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:09,967 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531949777_0001_1_00 for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] numStartedSources: 1 numSources: 1
2016-05-18 00:39:09,967 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] with distanceFromRoot: 1
2016-05-18 00:39:09,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531949777_0001_1_01, startRequestedTime=1463531949967, startedTime=1463531949967
2016-05-18 00:39:09,968 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:09,969 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in FD011450C4FA4A3A92E2266968E085EF
2016-05-18 00:39:09,969 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,969 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D80869F0FFA344B9A3F11AA006776E82 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:09,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_STARTED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, taskId=task_1463531949777_0001_1_00_000000, scheduledTime=1463531949969, launchTime=1463531949969
2016-05-18 00:39:09,970 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:09,971 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: FD011450C4FA4A3A92E2266968E085EF in vertex: D80869F0FFA344B9A3F11AA006776E82 numBipartiteSourceTasks: 1
2016-05-18 00:39:09,972 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:09,973 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531949777_0001, with dag id: dag_1463531949777_0001_1
2016-05-18 00:39:09,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531949777_0001_00_000001, launchTime=1463531949974
2016-05-18 00:39:09,975 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949777_0001_00_000001
2016-05-18 00:39:09,978 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949777_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:09,978 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531949777_0001_1_00_000000_0] started. Is using containerId: [container_1463531949777_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:09,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0, startTime=1463531949978, containerId=container_1463531949777_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:09 Starting to run new task attempt: attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:09,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:09,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:09,979 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:09,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: FD011450C4FA4A3A92E2266968E085EF, VertexParallelism: 1, TaskAttemptID:attempt_1463531949777_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BA3FA97452F84C97B1102E8DDCEF4AD9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D80869F0FFA344B9A3F11AA006776E82, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:09,980 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:09,981 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:09,986 INFO input.MRInput (MRInput.java:initialize(440)) - BA3FA97452F84C97B1102E8DDCEF4AD9 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:09,986 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531949777_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:09,987 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:09,987 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: BA3FA97452F84C97B1102E8DDCEF4AD9
2016-05-18 00:39:09,989 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531949777_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:09,992 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD011450C4FA4A3A92E2266968E085EF, ordinal: 0
2016-05-18 00:39:09,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD011450C4FA4A3A92E2266968E085EF, mem on start (mb), free: 93, total: 341, max: 672
2016-05-18 00:39:09,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:09,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:09,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:09,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:09,993 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[BA3FA97452F84C97B1102E8DDCEF4AD9:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D80869F0FFA344B9A3F11AA006776E82:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:09,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:09,994 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:09,994 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:09,994 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[BA3FA97452F84C97B1102E8DDCEF4AD9:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D80869F0FFA344B9A3F11AA006776E82:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:09,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:09,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BA3FA97452F84C97B1102E8DDCEF4AD9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:09,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:09,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:09,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BA3FA97452F84C97B1102E8DDCEF4AD9
2016-05-18 00:39:09,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:09,996 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:09,999 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,000 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: BA3FA97452F84C97B1102E8DDCEF4AD9
2016-05-18 00:39:10,000 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: D502F6638B4E4E10972D6B0D23538B91
2016-05-18 00:39:10,001 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D502F6638B4E4E10972D6B0D23538B91
2016-05-18 00:39:10,001 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:10,001 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - BA3FA97452F84C97B1102E8DDCEF4AD9 initialized RecordReader from event
2016-05-18 00:39:10,003 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D80869F0FFA344B9A3F11AA006776E82 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@35fe3a7e, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@45b241e3, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:10,003 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D80869F0FFA344B9A3F11AA006776E82: , UsingHashComparator=false
2016-05-18 00:39:10,024 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:10,025 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:10,025 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D80869F0FFA344B9A3F11AA006776E82: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:10,025 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D80869F0FFA344B9A3F11AA006776E82: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:10,025 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,026 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD011450C4FA4A3A92E2266968E085EF, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:10,031 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:10,031 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD011450C4FA4A3A92E2266968E085EF, mem on close (mb), free: 198, total: 342, max: 672
2016-05-18 00:39:10,032 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D80869F0FFA344B9A3F11AA006776E82: Starting flush of map output
2016-05-18 00:39:10,032 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D80869F0FFA344B9A3F11AA006776E82: Span0.length = 37, perItem = 24
2016-05-18 00:39:10,034 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D80869F0FFA344B9A3F11AA006776E82: done sorting span=0, length=37, time=2
2016-05-18 00:39:10,034 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D80869F0FFA344B9A3F11AA006776E82: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:10,035 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D80869F0FFA344B9A3F11AA006776E82: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-local-dir/output/attempt_1463531949777_0001_1_00_000000_0_10089_0/file.out
2016-05-18 00:39:10,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:10 Completed running task attempt: attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:10,039 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:10,040 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531949777_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:10,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531949777_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531950026, Process_Duration=5, Process_End_Time=1463531950031, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:10,041 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531949777_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,041 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,041 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:10,041 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, taskAttemptId=attempt_1463531949777_0001_1_00_000000_0, creationTime=1463531949970, allocationTime=1463531949973, startTime=1463531949978, finishTime=1463531950041, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:10,042 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949777_0001_00_000001
2016-05-18 00:39:10,042 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531949777_0001_1_00_000000_0
2016-05-18 00:39:10,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_FINISHED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, taskId=task_1463531949777_0001_1_00_000000, startTime=1463531949978, finishTime=1463531950042, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463531949777_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531950026, Process_Duration=5, Process_End_Time=1463531950031, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:10,044 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:10,044 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:10,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_FINISHED]: vertexName=FD011450C4FA4A3A92E2266968E085EF, vertexId=vertex_1463531949777_0001_1_00, initRequestedTime=1463531949948, initedTime=1463531949966, startRequestedTime=1463531949950, startedTime=1463531949966, finishTime=1463531950044, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531950026, Process_Duration=5, Process_End_Time=1463531950031, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531949978, firstTasksToStart=[ task_1463531949777_0001_1_00_000000 ], lastTaskFinishTime=1463531950041, lastTasksToFinish=[ task_1463531949777_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531949777_0001_1_00_000000 ], longestDurationTasks=[ task_1463531949777_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:10,045 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:10,045 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] attempt: attempt_1463531949777_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:10,046 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D80869F0FFA344B9A3F11AA006776E82
2016-05-18 00:39:10,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531949777_0001_00_000001, stoppedTime=1463531950045, exitStatus=0
2016-05-18 00:39:10,046 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=FD011450C4FA4A3A92E2266968E085EF, destinationVertexName=D80869F0FFA344B9A3F11AA006776E82. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:10,046 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:10,046 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:10,046 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,047 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531949777_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:10,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_STARTED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, taskId=task_1463531949777_0001_1_01_000000, scheduledTime=1463531950047, launchTime=1463531950047
2016-05-18 00:39:10,047 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:10,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531949777_0001_00_000002, launchTime=1463531950049
2016-05-18 00:39:10,049 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949777_0001_00_000002
2016-05-18 00:39:10,050 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949777_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531949777_0001_1_01_000000_0] started. Is using containerId: [container_1463531949777_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:10,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0, startTime=1463531950050, containerId=container_1463531949777_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:10 Starting to run new task attempt: attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,051 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,051 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,051 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:10,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: D80869F0FFA344B9A3F11AA006776E82, VertexParallelism: 1, TaskAttemptID:attempt_1463531949777_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FD011450C4FA4A3A92E2266968E085EF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F2E1BB218D24CA391E4CEEF432F0513, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:10,051 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:10,052 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,052 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531949777_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:10,053 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949777_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:10,052 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531949777_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:10,053 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531949777_0001_00_000001, Exiting
2016-05-18 00:39:10,053 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531949777_0001_00_000001
2016-05-18 00:39:10,053 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531949777_0001_00_000001
2016-05-18 00:39:10,054 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531949777_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:10,056 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531949777_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:10,056 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531949777_0001_00_000001 completed successfully
2016-05-18 00:39:10,057 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:10,059 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:10,067 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D80869F0FFA344B9A3F11AA006776E82, ordinal: 1
2016-05-18 00:39:10,067 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D80869F0FFA344B9A3F11AA006776E82, mem on start (mb), free: 193, total: 342, max: 672
2016-05-18 00:39:10,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:10,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:10,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:10,071 INFO output.MROutput (MROutput.java:initialize(421)) - 2F2E1BB218D24CA391E4CEEF432F0513: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:10,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:10,071 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[2F2E1BB218D24CA391E4CEEF432F0513:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [FD011450C4FA4A3A92E2266968E085EF:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:10,072 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:10,072 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:10,072 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:10,072 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[2F2E1BB218D24CA391E4CEEF432F0513:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [FD011450C4FA4A3A92E2266968E085EF:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:10,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:10,073 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - FD011450C4FA4A3A92E2266968E085EF: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:10,073 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - FD011450C4FA4A3A92E2266968E085EF: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:10,074 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: FD011450C4FA4A3A92E2266968E085EF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:10,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FD011450C4FA4A3A92E2266968E085EF
2016-05-18 00:39:10,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FD011450C4FA4A3A92E2266968E085EF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:10,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:10,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:10,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:10,079 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,079 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FD011450C4FA4A3A92E2266968E085EF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:10,081 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531949777_0001_1_00_000000_0_10089} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531950081, TimeTaken=0, Rate=0.00 MB/s
2016-05-18 00:39:10,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : FD011450C4FA4A3A92E2266968E085EF
2016-05-18 00:39:10,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:10,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: FD011450C4FA4A3A92E2266968E085EF, wasInterrupted=false
2016-05-18 00:39:10,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:10,083 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:10,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001_wd/localmode-local-dir/output/attempt_1463531949777_0001_1_00_000000_0_10089/file.out, len=732, isLocal=true
2016-05-18 00:39:10,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:10,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:10,085 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:10,085 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:10,085 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:10,085 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : FD011450C4FA4A3A92E2266968E085EF
2016-05-18 00:39:10,086 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - FD011450C4FA4A3A92E2266968E085EF: Shuffle Runner thread complete
2016-05-18 00:39:10,086 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: D502F6638B4E4E10972D6B0D23538B91
2016-05-18 00:39:10,086 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"], id: 2F2E1BB218D24CA391E4CEEF432F0513
2016-05-18 00:39:10,086 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"]
2016-05-18 00:39:10,086 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D502F6638B4E4E10972D6B0D23538B91, for 1 inputs
2016-05-18 00:39:10,086 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D80869F0FFA344B9A3F11AA006776E82, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:10,087 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - FD011450C4FA4A3A92E2266968E085EF: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:10,093 INFO output.MROutput (MROutput.java:close(526)) - 2F2E1BB218D24CA391E4CEEF432F0513 closed
2016-05-18 00:39:10,094 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531949777_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:10,094 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,094 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D80869F0FFA344B9A3F11AA006776E82, mem on close (mb), free: 190, total: 342, max: 672
2016-05-18 00:39:10,094 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: FD011450C4FA4A3A92E2266968E085EF
2016-05-18 00:39:10,094 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - FD011450C4FA4A3A92E2266968E085EF: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:10,094 INFO output.MROutput (MROutput.java:close(526)) - 2F2E1BB218D24CA391E4CEEF432F0513 closed
2016-05-18 00:39:10,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:10 Completed running task attempt: attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,095 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:10,096 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531949777_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:10,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531949777_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531950086, Process_Duration=8, Process_End_Time=1463531950094, Read_Duration=5, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:10,098 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531949777_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, taskAttemptId=attempt_1463531949777_0001_1_01_000000_0, creationTime=1463531950047, allocationTime=1463531950048, startTime=1463531950050, finishTime=1463531950098, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:10,098 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,099 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:10,099 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,099 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531949777_0001_1_01_000000_0
2016-05-18 00:39:10,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:TASK_FINISHED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, taskId=task_1463531949777_0001_1_01_000000, startTime=1463531950050, finishTime=1463531950099, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463531949777_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531950086, Process_Duration=8, Process_End_Time=1463531950094, Read_Duration=5, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:10,100 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531949777_0001_00_000002
2016-05-18 00:39:10,100 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:10,100 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82]
2016-05-18 00:39:10,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:VERTEX_FINISHED]: vertexName=D80869F0FFA344B9A3F11AA006776E82, vertexId=vertex_1463531949777_0001_1_01, initRequestedTime=1463531949950, initedTime=1463531949959, startRequestedTime=1463531949967, startedTime=1463531949967, finishTime=1463531950100, timeTaken=133, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531950086, Process_Duration=8, Process_End_Time=1463531950094, Read_Duration=5, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531950050, firstTasksToStart=[ task_1463531949777_0001_1_01_000000 ], lastTaskFinishTime=1463531950098, lastTasksToFinish=[ task_1463531949777_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531949777_0001_1_01_000000 ], longestDurationTasks=[ task_1463531949777_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:10,101 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:10,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531949777_0001_00_000002, stoppedTime=1463531950101, exitStatus=0
2016-05-18 00:39:10,102 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:10,102 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,102 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531949777_0001_1_00 [FD011450C4FA4A3A92E2266968E085EF]
2016-05-18 00:39:10,102 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531949777_0001_1
2016-05-18 00:39:10,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531949777_0001_1
2016-05-18 00:39:10,102 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 2F2E1BB218D24CA391E4CEEF432F0513 for vertex: vertex_1463531949777_0001_1_01 [D80869F0FFA344B9A3F11AA006776E82], outputName: 2F2E1BB218D24CA391E4CEEF432F0513
2016-05-18 00:39:10,104 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949777_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:10,104 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531949777_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:10,110 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:2F2E1BB218D24CA391E4CEEF432F0513 of vertex/vertexGroup:D80869F0FFA344B9A3F11AA006776E82 isVertexGroupOutput:false
2016-05-18 00:39:10,110 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,111 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531949777_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:10,111 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531949777_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:10,112 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531949777_0001_00_000002, Exiting
2016-05-18 00:39:10,112 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531949777_0001_00_000002
2016-05-18 00:39:10,112 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531949777_0001_00_000002
2016-05-18 00:39:10,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531949777_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:10,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531949777_0001_00_000002 completed successfully
2016-05-18 00:39:10,114 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531949777_0001_1, queueSize=0
2016-05-18 00:39:10,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531949777_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531949777_0001_1, startTime=1463531949947, finishTime=1463531950110, timeTaken=163, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=290, AM_GC_TIME_MILLIS=9, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063900112, Process_Duration=13, Process_End_Time=2927063900125, Read_Duration=6, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:10,115 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531949777_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:10,115 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531949777_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:10 Completed Dag: dag_1463531949777_0001_1
2016-05-18 00:39:10,116 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531949777_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:10,116 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531949777_0001_1_post
2016-05-18 00:39:10,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,116 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:10,116 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:10,117 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false, with id=dag_1463531949777_0001_1
2016-05-18 00:39:10,117 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3] comprehensivesort/(1/1) ...ive/false_true_true_false, with id=dag_1463531949777_0001_1
2016-05-18 00:39:10,125 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1463531949777_0001
2016-05-18 00:39:10,127 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:10,127 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:10,127 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:10,127 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:10,128 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:10,129 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:10,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:10,129 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:10,130 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:10,130 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:10,130 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:10,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:10,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:10,131 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:10,136 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FBB52C6ADAE44CB998FED661D72052BA/C4E904D2C9914358906664DF3FEF59C3/.tez/application_1463531949777_0001
2016-05-18 00:39:10,136 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false/_temporary
2016-05-18 00:39:10,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.383
2016-05-18 00:39:10,229 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:10,229 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:10,230 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:10,230 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:10,280 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:39:10,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-18 00:39:10,294 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:10,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:10,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:10,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:10,328 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true
2016-05-18 00:39:10,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:10,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"]
2016-05-18 00:39:10,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:10,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:10,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:10,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_false_true
2016-05-18 00:39:10,385 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:10,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:10,402 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:10,402 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:10,403 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:10,411 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001 doesn't exist and is created
2016-05-18 00:39:10,429 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:10,429 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:10,430 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd
2016-05-18 00:39:10,442 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531950402_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:10,442 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:10,442 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:10,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:10,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:10,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:10,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:10,445 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:10,447 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:10,447 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:10,448 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:10,448 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:10,449 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:10,449 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:10,449 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:10,450 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:10,450 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:10,450 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:10,451 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:10,451 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:10,453 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:10,454 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:10,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:10,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-log-dir
2016-05-18 00:39:10,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-log-dir/history.txt.appattempt_1463531950402_0001_000000, maxErrors=10
2016-05-18 00:39:10,456 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:10,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531950402_0001_000000, appSubmitTime=1463531950441, launchTime=1463531950441
2016-05-18 00:39:10,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:10,461 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40513
2016-05-18 00:39:10,461 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531950402
2016-05-18 00:39:10,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531950402_0001_000000, startTime=1463531950464
2016-05-18 00:39:10,464 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:10,529 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:10,529 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:10,530 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1463531950402_0001, dagName=[DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-05-18 00:39:10,546 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-05-18 00:39:10,546 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:10,547 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531950402_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-log-dir/dag_1463531950402_0001_1.dot
2016-05-18 00:39:10,547 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-log-dir/dag_1463531950402_0001_1-tez-dag.pb.txt
2016-05-18 00:39:10,553 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531950402_0001_1
2016-05-18 00:39:10,554 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,554 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-05-18 00:39:10 Running Dag: dag_1463531950402_0001_1
2016-05-18 00:39:10,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531950402_0001_1, submitTime=1463531950546
2016-05-18 00:39:10,565 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001/tez-conf.pb" } size: 36903 timestamp: 1463531950000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:10,565 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,566 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:10,567 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001/tez-conf.pb" } size: 36903 timestamp: 1463531950000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:10,567 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,568 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:10,569 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:10,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531950402_0001_1, initTime=1463531950569
2016-05-18 00:39:10,569 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950402_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:10,570 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:10,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:DAG_STARTED]: dagID=dag_1463531950402_0001_1, startTime=1463531950572
2016-05-18 00:39:10,572 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950402_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:10,572 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B1423BC1CA4E4FD197DC2A374FA86B13 : {808A279F54F14040AA27F8AFC4EFB7E4={InputName=808A279F54F14040AA27F8AFC4EFB7E4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:10,572 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 808A279F54F14040AA27F8AFC4EFB7E4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:10,572 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,572 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,572 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,573 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,573 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:10,573 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 808A279F54F14040AA27F8AFC4EFB7E4 on vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,573 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,573 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1463531950402_0001, dagName=[DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-05-18 00:39:10,576 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 808A279F54F14040AA27F8AFC4EFB7E4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:10,577 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:10,577 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,577 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:10,578 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,578 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:10,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531950402_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:10,578 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A], numAdditionalOutputs=1
2016-05-18 00:39:10,579 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A234CD71612A49DA955CC6684B08E7D1, vertex=vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:10,581 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F9871060E6EB45D39ED24252671F538A:A234CD71612A49DA955CC6684B08E7D1 using oldmapred API
2016-05-18 00:39:10,581 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A234CD71612A49DA955CC6684B08E7D1, vertexName=F9871060E6EB45D39ED24252671F538A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:10,578 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:10,581 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:10,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9871060E6EB45D39ED24252671F538A, vertexId=vertex_1463531950402_0001_1_01, initRequestedTime=1463531950573, initedTime=1463531950578, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:10,582 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:10,582 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 808A279F54F14040AA27F8AFC4EFB7E4 on vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,583 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:10,584 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {808A279F54F14040AA27F8AFC4EFB7E4=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:10,586 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] parallelism set to 1
2016-05-18 00:39:10,586 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:10,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, vertexId=vertex_1463531950402_0001_1_00, initRequestedTime=1463531950572, initedTime=1463531950586, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:10,587 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] with distanceFromRoot: 0
2016-05-18 00:39:10,587 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:10,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531950402_0001_1_00, startRequestedTime=1463531950573, startedTime=1463531950587
2016-05-18 00:39:10,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531950402_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:10,587 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:10,587 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531950402_0001_1_00 for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] numStartedSources: 1 numSources: 1
2016-05-18 00:39:10,588 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] with distanceFromRoot: 1
2016-05-18 00:39:10,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531950402_0001_1_01, startRequestedTime=1463531950587, startedTime=1463531950588
2016-05-18 00:39:10,588 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:10,591 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B1423BC1CA4E4FD197DC2A374FA86B13
2016-05-18 00:39:10,591 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F9871060E6EB45D39ED24252671F538A with 0 source tasks and 1 pending tasks
2016-05-18 00:39:10,591 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,592 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B1423BC1CA4E4FD197DC2A374FA86B13 in vertex: F9871060E6EB45D39ED24252671F538A numBipartiteSourceTasks: 1
2016-05-18 00:39:10,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_STARTED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, taskId=task_1463531950402_0001_1_00_000000, scheduledTime=1463531950592, launchTime=1463531950592
2016-05-18 00:39:10,592 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,593 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:10,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531950402_0001_00_000001, launchTime=1463531950595
2016-05-18 00:39:10,595 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950402_0001_00_000001
2016-05-18 00:39:10,596 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950402_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,596 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531950402_0001_1_00_000000_0] started. Is using containerId: [container_1463531950402_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:10,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0, startTime=1463531950596, containerId=container_1463531950402_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:10 Starting to run new task attempt: attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,597 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,597 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,597 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:10,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: B1423BC1CA4E4FD197DC2A374FA86B13, VertexParallelism: 1, TaskAttemptID:attempt_1463531950402_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=808A279F54F14040AA27F8AFC4EFB7E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F9871060E6EB45D39ED24252671F538A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:10,597 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:10,598 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,598 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531950402_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:10,599 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531950402_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:10,599 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531950402_0001, with dag id: dag_1463531950402_0001_1
2016-05-18 00:39:10,603 INFO input.MRInput (MRInput.java:initialize(440)) - 808A279F54F14040AA27F8AFC4EFB7E4 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:10,604 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 808A279F54F14040AA27F8AFC4EFB7E4
2016-05-18 00:39:10,604 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:10,610 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1423BC1CA4E4FD197DC2A374FA86B13, ordinal: 0
2016-05-18 00:39:10,610 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1423BC1CA4E4FD197DC2A374FA86B13, mem on start (mb), free: 93, total: 342, max: 672
2016-05-18 00:39:10,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:10,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:10,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:10,611 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[808A279F54F14040AA27F8AFC4EFB7E4:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F9871060E6EB45D39ED24252671F538A:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:10,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:10,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:10,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:10,611 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[808A279F54F14040AA27F8AFC4EFB7E4:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F9871060E6EB45D39ED24252671F538A:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 808A279F54F14040AA27F8AFC4EFB7E4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:10,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 808A279F54F14040AA27F8AFC4EFB7E4
2016-05-18 00:39:10,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:10,612 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,613 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:10,613 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 808A279F54F14040AA27F8AFC4EFB7E4 initialized RecordReader from event
2016-05-18 00:39:10,615 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,615 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 808A279F54F14040AA27F8AFC4EFB7E4
2016-05-18 00:39:10,615 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: B21AEE0F019845F2B5252DB908F090BF
2016-05-18 00:39:10,616 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B21AEE0F019845F2B5252DB908F090BF
2016-05-18 00:39:10,617 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F9871060E6EB45D39ED24252671F538A using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@45ae4c6a, comparator=cascading.tuple.hadoop.util.TupleComparator@361ae6e3, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:10,617 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F9871060E6EB45D39ED24252671F538A: , UsingHashComparator=false
2016-05-18 00:39:10,639 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:10,640 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:10,640 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F9871060E6EB45D39ED24252671F538A: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:10,640 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F9871060E6EB45D39ED24252671F538A: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:10,640 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,640 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1423BC1CA4E4FD197DC2A374FA86B13, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:10,646 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,646 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1423BC1CA4E4FD197DC2A374FA86B13, mem on close (mb), free: 197, total: 342, max: 672
2016-05-18 00:39:10,647 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F9871060E6EB45D39ED24252671F538A: Starting flush of map output
2016-05-18 00:39:10,647 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F9871060E6EB45D39ED24252671F538A: Span0.length = 37, perItem = 17
2016-05-18 00:39:10,648 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F9871060E6EB45D39ED24252671F538A: done sorting span=0, length=37, time=1
2016-05-18 00:39:10,649 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F9871060E6EB45D39ED24252671F538A: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:10,650 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F9871060E6EB45D39ED24252671F538A: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-local-dir/output/attempt_1463531950402_0001_1_00_000000_0_10095_0/file.out
2016-05-18 00:39:10,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:10 Completed running task attempt: attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,653 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:10,654 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531950402_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:10,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531950402_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531950641, Process_Duration=5, Process_End_Time=1463531950646, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:10,656 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531950402_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,656 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,656 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:10,656 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,657 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950402_0001_00_000001
2016-05-18 00:39:10,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, taskAttemptId=attempt_1463531950402_0001_1_00_000000_0, creationTime=1463531950592, allocationTime=1463531950594, startTime=1463531950596, finishTime=1463531950657, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:10,657 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531950402_0001_1_00_000000_0
2016-05-18 00:39:10,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_FINISHED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, taskId=task_1463531950402_0001_1_00_000000, startTime=1463531950596, finishTime=1463531950657, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463531950402_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531950641, Process_Duration=5, Process_End_Time=1463531950646, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:10,658 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:10,658 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_FINISHED]: vertexName=B1423BC1CA4E4FD197DC2A374FA86B13, vertexId=vertex_1463531950402_0001_1_00, initRequestedTime=1463531950572, initedTime=1463531950586, startRequestedTime=1463531950573, startedTime=1463531950587, finishTime=1463531950658, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531950641, Process_Duration=5, Process_End_Time=1463531950646, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531950596, firstTasksToStart=[ task_1463531950402_0001_1_00_000000 ], lastTaskFinishTime=1463531950657, lastTasksToFinish=[ task_1463531950402_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531950402_0001_1_00_000000 ], longestDurationTasks=[ task_1463531950402_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:10,659 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:10,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531950402_0001_00_000001, stoppedTime=1463531950660, exitStatus=0
2016-05-18 00:39:10,660 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] attempt: attempt_1463531950402_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:10,660 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F9871060E6EB45D39ED24252671F538A
2016-05-18 00:39:10,660 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:10,661 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,661 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B1423BC1CA4E4FD197DC2A374FA86B13, destinationVertexName=F9871060E6EB45D39ED24252671F538A. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:10,661 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531950402_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:10,661 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_STARTED]: vertexName=F9871060E6EB45D39ED24252671F538A, taskId=task_1463531950402_0001_1_01_000000, scheduledTime=1463531950661, launchTime=1463531950661
2016-05-18 00:39:10,662 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531950402_0001_00_000002, launchTime=1463531950663
2016-05-18 00:39:10,664 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950402_0001_00_000002
2016-05-18 00:39:10,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531950402_0001_1_01_000000_0] started. Is using containerId: [container_1463531950402_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:10,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9871060E6EB45D39ED24252671F538A, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0, startTime=1463531950664, containerId=container_1463531950402_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:10,665 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950402_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10 Starting to run new task attempt: attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,665 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,666 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:10,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: F9871060E6EB45D39ED24252671F538A, VertexParallelism: 1, TaskAttemptID:attempt_1463531950402_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1423BC1CA4E4FD197DC2A374FA86B13, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A234CD71612A49DA955CC6684B08E7D1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:10,667 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:10,667 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,668 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531950402_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:10,668 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950402_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:10,668 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531950402_0001_00_000001, Exiting
2016-05-18 00:39:10,668 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531950402_0001_00_000001
2016-05-18 00:39:10,668 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531950402_0001_00_000001
2016-05-18 00:39:10,668 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531950402_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,669 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531950402_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:10,670 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531950402_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:10,670 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531950402_0001_00_000001 completed successfully
2016-05-18 00:39:10,670 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:10,672 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:10,678 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9871060E6EB45D39ED24252671F538A, ordinal: 1
2016-05-18 00:39:10,678 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9871060E6EB45D39ED24252671F538A, mem on start (mb), free: 192, total: 342, max: 672
2016-05-18 00:39:10,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:10,679 INFO output.MROutput (MROutput.java:initialize(421)) - A234CD71612A49DA955CC6684B08E7D1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:10,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:10,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:10,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:10,680 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A234CD71612A49DA955CC6684B08E7D1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [B1423BC1CA4E4FD197DC2A374FA86B13:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:10,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:10,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:10,681 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:10,681 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A234CD71612A49DA955CC6684B08E7D1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [B1423BC1CA4E4FD197DC2A374FA86B13:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:10,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:10,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B1423BC1CA4E4FD197DC2A374FA86B13 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:10,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:10,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:10,682 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B1423BC1CA4E4FD197DC2A374FA86B13: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:10,683 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B1423BC1CA4E4FD197DC2A374FA86B13: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:10,684 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B1423BC1CA4E4FD197DC2A374FA86B13 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:10,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B1423BC1CA4E4FD197DC2A374FA86B13
2016-05-18 00:39:10,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:10,685 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,686 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1423BC1CA4E4FD197DC2A374FA86B13: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:10,689 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531950402_0001_1_00_000000_0_10095} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531950688, TimeTaken=2, Rate=0.28 MB/s
2016-05-18 00:39:10,689 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B1423BC1CA4E4FD197DC2A374FA86B13
2016-05-18 00:39:10,689 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:10,690 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:10,692 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: B21AEE0F019845F2B5252DB908F090BF
2016-05-18 00:39:10,692 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"], id: A234CD71612A49DA955CC6684B08E7D1
2016-05-18 00:39:10,692 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"]
2016-05-18 00:39:10,692 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B21AEE0F019845F2B5252DB908F090BF, for 1 inputs
2016-05-18 00:39:10,693 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B1423BC1CA4E4FD197DC2A374FA86B13, wasInterrupted=false
2016-05-18 00:39:10,693 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:10,694 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:10,694 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001_wd/localmode-local-dir/output/attempt_1463531950402_0001_1_00_000000_0_10095/file.out, len=593, isLocal=true
2016-05-18 00:39:10,695 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:10,695 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:10,695 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:10,695 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:10,695 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9871060E6EB45D39ED24252671F538A, all 1 inputs ready in: 00:00:00.003
2016-05-18 00:39:10,696 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B1423BC1CA4E4FD197DC2A374FA86B13
2016-05-18 00:39:10,696 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B1423BC1CA4E4FD197DC2A374FA86B13: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:10,697 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B1423BC1CA4E4FD197DC2A374FA86B13: Shuffle Runner thread complete
2016-05-18 00:39:10,703 INFO output.MROutput (MROutput.java:close(526)) - A234CD71612A49DA955CC6684B08E7D1 closed
2016-05-18 00:39:10,704 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531950402_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:10,704 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,705 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9871060E6EB45D39ED24252671F538A, mem on close (mb), free: 189, total: 342, max: 672
2016-05-18 00:39:10,705 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B1423BC1CA4E4FD197DC2A374FA86B13
2016-05-18 00:39:10,705 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B1423BC1CA4E4FD197DC2A374FA86B13: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:10,705 INFO output.MROutput (MROutput.java:close(526)) - A234CD71612A49DA955CC6684B08E7D1 closed
2016-05-18 00:39:10,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:10 Completed running task attempt: attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,707 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:10,707 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531950402_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:10,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531950402_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531950696, Process_Duration=8, Process_End_Time=1463531950704, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:10,709 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531950402_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,709 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:10,709 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:10,709 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9871060E6EB45D39ED24252671F538A, taskAttemptId=attempt_1463531950402_0001_1_01_000000_0, creationTime=1463531950662, allocationTime=1463531950662, startTime=1463531950664, finishTime=1463531950709, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:10,710 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531950402_0001_1_01_000000_0
2016-05-18 00:39:10,710 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950402_0001_00_000002
2016-05-18 00:39:10,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:TASK_FINISHED]: vertexName=F9871060E6EB45D39ED24252671F538A, taskId=task_1463531950402_0001_1_01_000000, startTime=1463531950664, finishTime=1463531950710, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463531950402_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531950696, Process_Duration=8, Process_End_Time=1463531950704, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:10,711 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:10,711 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A]
2016-05-18 00:39:10,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:VERTEX_FINISHED]: vertexName=F9871060E6EB45D39ED24252671F538A, vertexId=vertex_1463531950402_0001_1_01, initRequestedTime=1463531950573, initedTime=1463531950578, startRequestedTime=1463531950587, startedTime=1463531950588, finishTime=1463531950711, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531950696, Process_Duration=8, Process_End_Time=1463531950704, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531950664, firstTasksToStart=[ task_1463531950402_0001_1_01_000000 ], lastTaskFinishTime=1463531950709, lastTasksToFinish=[ task_1463531950402_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531950402_0001_1_01_000000 ], longestDurationTasks=[ task_1463531950402_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:10,712 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:10,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531950402_0001_00_000002, stoppedTime=1463531950712, exitStatus=0
2016-05-18 00:39:10,712 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:10,712 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,712 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531950402_0001_1_00 [B1423BC1CA4E4FD197DC2A374FA86B13]
2016-05-18 00:39:10,713 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531950402_0001_1
2016-05-18 00:39:10,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531950402_0001_1
2016-05-18 00:39:10,713 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A234CD71612A49DA955CC6684B08E7D1 for vertex: vertex_1463531950402_0001_1_01 [F9871060E6EB45D39ED24252671F538A], outputName: A234CD71612A49DA955CC6684B08E7D1
2016-05-18 00:39:10,716 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950402_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:10,716 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531950402_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:10,720 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A234CD71612A49DA955CC6684B08E7D1 of vertex/vertexGroup:F9871060E6EB45D39ED24252671F538A isVertexGroupOutput:false
2016-05-18 00:39:10,720 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:10,720 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531950402_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:10,720 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950402_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:10,721 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531950402_0001_00_000002, Exiting
2016-05-18 00:39:10,721 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531950402_0001_00_000002
2016-05-18 00:39:10,721 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531950402_0001_00_000002
2016-05-18 00:39:10,721 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531950402_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:10,721 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531950402_0001_00_000002 completed successfully
2016-05-18 00:39:10,722 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531950402_0001_1, queueSize=0
2016-05-18 00:39:10,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950402_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531950402_0001_1, startTime=1463531950570, finishTime=1463531950720, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=290, AM_GC_TIME_MILLIS=8, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063901337, Process_Duration=13, Process_End_Time=2927063901350, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=4, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:10,723 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531950402_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:10,723 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950402_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:10 Completed Dag: dag_1463531950402_0001_1
2016-05-18 00:39:10,723 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531950402_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:10,723 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531950402_0001_1_post
2016-05-18 00:39:10,723 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:10,723 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:10,723 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:10,723 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true, with id=dag_1463531950402_0001_1
2016-05-18 00:39:10,724 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26] comprehensivesort/(1/1) ...ive/false_null_false_true, with id=dag_1463531950402_0001_1
2016-05-18 00:39:10,734 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1463531950402_0001
2016-05-18 00:39:10,736 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:10,736 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:10,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:10,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:10,736 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:10,737 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:10,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:10,738 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:10,738 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:10,738 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:10,738 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:10,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:10,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:10,739 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:10,741 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/DF58C8EBD12C4321B96180CD6FE032FD/C3F0E53BBF2944D088A3603F55ED6A26/.tez/application_1463531950402_0001
2016-05-18 00:39:10,742 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true/_temporary
2016-05-18 00:39:10,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.361
2016-05-18 00:39:10,812 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:10,813 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:10,847 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:10,847 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:10,852 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-18 00:39:10,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-18 00:39:10,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:10,867 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:10,867 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:10,882 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:10,888 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true
2016-05-18 00:39:10,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:10,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"]
2016-05-18 00:39:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:10,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_false_true
2016-05-18 00:39:10,938 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:10,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:10,954 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:10,955 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:10,955 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:10,964 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001 doesn't exist and is created
2016-05-18 00:39:10,981 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:10,981 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:10,982 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd
2016-05-18 00:39:10,992 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531950955_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:10,993 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:10,993 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:10,995 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:10,995 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:10,995 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:10,995 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:10,996 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:10,997 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:10,997 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:10,998 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:10,998 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:10,998 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:10,998 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:10,999 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:10,999 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:10,999 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:10,999 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:11,000 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:11,000 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:11,002 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:11,002 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:11,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:11,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-log-dir
2016-05-18 00:39:11,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-log-dir/history.txt.appattempt_1463531950955_0001_000000, maxErrors=10
2016-05-18 00:39:11,004 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:11,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531950955_0001_000000, appSubmitTime=1463531950992, launchTime=1463531950992
2016-05-18 00:39:11,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:11,014 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43261
2016-05-18 00:39:11,015 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531950955
2016-05-18 00:39:11,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531950955_0001_000000, startTime=1463531951017
2016-05-18 00:39:11,017 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:11,082 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:11,082 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:11,082 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1463531950955_0001, dagName=[AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-05-18 00:39:11,098 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-05-18 00:39:11,098 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:11,099 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531950955_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-log-dir/dag_1463531950955_0001_1.dot
2016-05-18 00:39:11,100 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-log-dir/dag_1463531950955_0001_1-tez-dag.pb.txt
2016-05-18 00:39:11,109 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531950955_0001_1
2016-05-18 00:39:11,109 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,109 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-05-18 00:39:11 Running Dag: dag_1463531950955_0001_1
2016-05-18 00:39:11,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531950955_0001_1, submitTime=1463531951098
2016-05-18 00:39:11,119 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001/tez-conf.pb" } size: 36903 timestamp: 1463531950000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:11,119 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,119 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:11,120 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001/tez-conf.pb" } size: 36903 timestamp: 1463531950000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:11,121 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,121 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:11,121 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:11,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531950955_0001_1, initTime=1463531951122
2016-05-18 00:39:11,122 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950955_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:11,123 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:11,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:DAG_STARTED]: dagID=dag_1463531950955_0001_1, startTime=1463531951123
2016-05-18 00:39:11,123 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950955_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:11,124 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: B549AB2A9CB84FD99631D30C13718C72 : {4C5CC88D518A4598A368A8C826E86C0D={InputName=4C5CC88D518A4598A368A8C826E86C0D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:11,124 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 4C5CC88D518A4598A368A8C826E86C0D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:11,125 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 4C5CC88D518A4598A368A8C826E86C0D on vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,125 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,128 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1463531950955_0001, dagName=[AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-05-18 00:39:11,130 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 4C5CC88D518A4598A368A8C826E86C0D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:11,130 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:11,130 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,130 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531950955_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:11,131 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B], numAdditionalOutputs=1
2016-05-18 00:39:11,131 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=A793F7D178534BD5989ACF974AC550B6, vertex=vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:11,132 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:11,132 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:11,132 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:11,133 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:11,133 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 4C5CC88D518A4598A368A8C826E86C0D on vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,133 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for F926D9CEA49648F489410244FE47491B:A793F7D178534BD5989ACF974AC550B6 using oldmapred API
2016-05-18 00:39:11,133 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=A793F7D178534BD5989ACF974AC550B6, vertexName=F926D9CEA49648F489410244FE47491B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:11,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F926D9CEA49648F489410244FE47491B, vertexId=vertex_1463531950955_0001_1_01, initRequestedTime=1463531951125, initedTime=1463531951131, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:11,134 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:11,136 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:11,136 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {4C5CC88D518A4598A368A8C826E86C0D=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:11,137 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] parallelism set to 1
2016-05-18 00:39:11,137 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:11,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, vertexId=vertex_1463531950955_0001_1_00, initRequestedTime=1463531951124, initedTime=1463531951137, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:11,137 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] with distanceFromRoot: 0
2016-05-18 00:39:11,137 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:11,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531950955_0001_1_00, startRequestedTime=1463531951125, startedTime=1463531951138
2016-05-18 00:39:11,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531950955_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:11,139 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:11,139 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531950955_0001_1_00 for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] numStartedSources: 1 numSources: 1
2016-05-18 00:39:11,139 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] with distanceFromRoot: 1
2016-05-18 00:39:11,139 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in B549AB2A9CB84FD99631D30C13718C72
2016-05-18 00:39:11,140 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,140 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531950955_0001_1_01, startRequestedTime=1463531951139, startedTime=1463531951140
2016-05-18 00:39:11,141 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:11,145 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: F926D9CEA49648F489410244FE47491B with 0 source tasks and 1 pending tasks
2016-05-18 00:39:11,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_STARTED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, taskId=task_1463531950955_0001_1_00_000000, scheduledTime=1463531951142, launchTime=1463531951142
2016-05-18 00:39:11,146 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: B549AB2A9CB84FD99631D30C13718C72 in vertex: F926D9CEA49648F489410244FE47491B numBipartiteSourceTasks: 1
2016-05-18 00:39:11,146 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,147 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:11,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531950955_0001_00_000001, launchTime=1463531951149
2016-05-18 00:39:11,149 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950955_0001_00_000001
2016-05-18 00:39:11,150 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531950955_0001_1_00_000000_0] started. Is using containerId: [container_1463531950955_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:11,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0, startTime=1463531951150, containerId=container_1463531950955_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:11,151 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950955_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11 Starting to run new task attempt: attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,151 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,151 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,152 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:11,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: B549AB2A9CB84FD99631D30C13718C72, VertexParallelism: 1, TaskAttemptID:attempt_1463531950955_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4C5CC88D518A4598A368A8C826E86C0D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F926D9CEA49648F489410244FE47491B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:11,153 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:11,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531950955_0001, with dag id: dag_1463531950955_0001_1
2016-05-18 00:39:11,154 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,155 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531950955_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:11,155 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531950955_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:11,158 INFO input.MRInput (MRInput.java:initialize(440)) - 4C5CC88D518A4598A368A8C826E86C0D using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:11,158 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:11,159 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 4C5CC88D518A4598A368A8C826E86C0D
2016-05-18 00:39:11,164 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B549AB2A9CB84FD99631D30C13718C72, ordinal: 0
2016-05-18 00:39:11,164 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B549AB2A9CB84FD99631D30C13718C72, mem on start (mb), free: 89, total: 342, max: 672
2016-05-18 00:39:11,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:11,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:11,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:11,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:11,164 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[4C5CC88D518A4598A368A8C826E86C0D:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [F926D9CEA49648F489410244FE47491B:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:11,165 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:11,165 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:11,165 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:11,165 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[4C5CC88D518A4598A368A8C826E86C0D:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [F926D9CEA49648F489410244FE47491B:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:11,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:11,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 4C5CC88D518A4598A368A8C826E86C0D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:11,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 4C5CC88D518A4598A368A8C826E86C0D
2016-05-18 00:39:11,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:11,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:11,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:11,167 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,168 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:11,168 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 4C5CC88D518A4598A368A8C826E86C0D initialized RecordReader from event
2016-05-18 00:39:11,170 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:11,170 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 4C5CC88D518A4598A368A8C826E86C0D
2016-05-18 00:39:11,170 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: D27ABD6D909E41B7ACB67F7C71EA6A02
2016-05-18 00:39:11,171 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D27ABD6D909E41B7ACB67F7C71EA6A02
2016-05-18 00:39:11,173 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - F926D9CEA49648F489410244FE47491B using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@606d2f39, comparator=cascading.tuple.hadoop.util.TupleComparator@19d16782, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:11,174 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for F926D9CEA49648F489410244FE47491B: , UsingHashComparator=false
2016-05-18 00:39:11,197 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:11,197 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:11,198 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for F926D9CEA49648F489410244FE47491B: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:11,198 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - F926D9CEA49648F489410244FE47491B: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:11,198 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:11,198 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B549AB2A9CB84FD99631D30C13718C72, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:11,203 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,203 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B549AB2A9CB84FD99631D30C13718C72, mem on close (mb), free: 193, total: 340, max: 672
2016-05-18 00:39:11,203 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - F926D9CEA49648F489410244FE47491B: Starting flush of map output
2016-05-18 00:39:11,204 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - F926D9CEA49648F489410244FE47491B: Span0.length = 37, perItem = 17
2016-05-18 00:39:11,204 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - F926D9CEA49648F489410244FE47491B: done sorting span=0, length=37, time=0
2016-05-18 00:39:11,204 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - F926D9CEA49648F489410244FE47491B: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:11,206 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - F926D9CEA49648F489410244FE47491B: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-local-dir/output/attempt_1463531950955_0001_1_00_000000_0_10101_0/file.out
2016-05-18 00:39:11,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:11 Completed running task attempt: attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,209 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:11,210 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531950955_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:11,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531950955_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531951199, Process_Duration=4, Process_End_Time=1463531951203, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:11,211 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531950955_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,211 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,212 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:11,212 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,212 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950955_0001_00_000001
2016-05-18 00:39:11,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, taskAttemptId=attempt_1463531950955_0001_1_00_000000_0, creationTime=1463531951146, allocationTime=1463531951148, startTime=1463531951150, finishTime=1463531951212, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:11,213 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531950955_0001_1_00_000000_0
2016-05-18 00:39:11,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_FINISHED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, taskId=task_1463531950955_0001_1_00_000000, startTime=1463531951150, finishTime=1463531951213, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463531950955_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531951199, Process_Duration=4, Process_End_Time=1463531951203, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:11,214 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:11,214 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_FINISHED]: vertexName=B549AB2A9CB84FD99631D30C13718C72, vertexId=vertex_1463531950955_0001_1_00, initRequestedTime=1463531951124, initedTime=1463531951137, startRequestedTime=1463531951125, startedTime=1463531951138, finishTime=1463531951214, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531951199, Process_Duration=4, Process_End_Time=1463531951203, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531951150, firstTasksToStart=[ task_1463531950955_0001_1_00_000000 ], lastTaskFinishTime=1463531951212, lastTasksToFinish=[ task_1463531950955_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531950955_0001_1_00_000000 ], longestDurationTasks=[ task_1463531950955_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:11,215 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:11,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531950955_0001_00_000001, stoppedTime=1463531951216, exitStatus=0
2016-05-18 00:39:11,216 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] attempt: attempt_1463531950955_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:11,216 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:11,216 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F926D9CEA49648F489410244FE47491B
2016-05-18 00:39:11,216 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=B549AB2A9CB84FD99631D30C13718C72, destinationVertexName=F926D9CEA49648F489410244FE47491B. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:11,216 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,216 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,217 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531950955_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:11,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_STARTED]: vertexName=F926D9CEA49648F489410244FE47491B, taskId=task_1463531950955_0001_1_01_000000, scheduledTime=1463531951217, launchTime=1463531951217
2016-05-18 00:39:11,217 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531950955_0001_00_000002, launchTime=1463531951219
2016-05-18 00:39:11,219 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950955_0001_00_000002
2016-05-18 00:39:11,220 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531950955_0001_1_01_000000_0] started. Is using containerId: [container_1463531950955_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:11,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F926D9CEA49648F489410244FE47491B, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0, startTime=1463531951220, containerId=container_1463531950955_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:11,221 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950955_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11 Starting to run new task attempt: attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,222 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,222 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,222 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:11,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: F926D9CEA49648F489410244FE47491B, VertexParallelism: 1, TaskAttemptID:attempt_1463531950955_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B549AB2A9CB84FD99631D30C13718C72, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A793F7D178534BD5989ACF974AC550B6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:11,223 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:11,223 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531950955_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:11,223 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950955_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:11,223 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531950955_0001_00_000001, Exiting
2016-05-18 00:39:11,224 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531950955_0001_00_000001
2016-05-18 00:39:11,224 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531950955_0001_00_000001
2016-05-18 00:39:11,224 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,225 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531950955_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,225 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531950955_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:11,225 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531950955_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:11,225 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531950955_0001_00_000001 completed successfully
2016-05-18 00:39:11,227 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:11,228 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:11,236 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F926D9CEA49648F489410244FE47491B, ordinal: 1
2016-05-18 00:39:11,236 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F926D9CEA49648F489410244FE47491B, mem on start (mb), free: 188, total: 340, max: 672
2016-05-18 00:39:11,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:11,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:11,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:11,240 INFO output.MROutput (MROutput.java:initialize(421)) - A793F7D178534BD5989ACF974AC550B6: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:11,240 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:11,240 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B549AB2A9CB84FD99631D30C13718C72:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [A793F7D178534BD5989ACF974AC550B6:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:11,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:11,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:11,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:11,241 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B549AB2A9CB84FD99631D30C13718C72:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [A793F7D178534BD5989ACF974AC550B6:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:11,241 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:11,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B549AB2A9CB84FD99631D30C13718C72 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:11,242 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - B549AB2A9CB84FD99631D30C13718C72: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:11,242 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - B549AB2A9CB84FD99631D30C13718C72: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:11,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:11,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:11,243 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: B549AB2A9CB84FD99631D30C13718C72 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:11,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B549AB2A9CB84FD99631D30C13718C72
2016-05-18 00:39:11,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:11,244 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,246 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B549AB2A9CB84FD99631D30C13718C72: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:11,247 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531950955_0001_1_00_000000_0_10101} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531951247, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:11,248 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : B549AB2A9CB84FD99631D30C13718C72
2016-05-18 00:39:11,248 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:11,248 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:11,249 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: D27ABD6D909E41B7ACB67F7C71EA6A02
2016-05-18 00:39:11,249 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"], id: A793F7D178534BD5989ACF974AC550B6
2016-05-18 00:39:11,249 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"]
2016-05-18 00:39:11,249 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D27ABD6D909E41B7ACB67F7C71EA6A02, for 1 inputs
2016-05-18 00:39:11,251 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: B549AB2A9CB84FD99631D30C13718C72, wasInterrupted=false
2016-05-18 00:39:11,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:11,253 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:11,253 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001_wd/localmode-local-dir/output/attempt_1463531950955_0001_1_00_000000_0_10101/file.out, len=593, isLocal=true
2016-05-18 00:39:11,254 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:11,254 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:11,254 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:11,254 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:11,255 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : B549AB2A9CB84FD99631D30C13718C72
2016-05-18 00:39:11,255 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - B549AB2A9CB84FD99631D30C13718C72: Shuffle Runner thread complete
2016-05-18 00:39:11,255 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F926D9CEA49648F489410244FE47491B, all 1 inputs ready in: 00:00:00.006
2016-05-18 00:39:11,256 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - B549AB2A9CB84FD99631D30C13718C72: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.TupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:11,260 INFO output.MROutput (MROutput.java:close(526)) - A793F7D178534BD5989ACF974AC550B6 closed
2016-05-18 00:39:11,260 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531950955_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:11,261 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,261 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F926D9CEA49648F489410244FE47491B, mem on close (mb), free: 185, total: 340, max: 672
2016-05-18 00:39:11,261 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: B549AB2A9CB84FD99631D30C13718C72
2016-05-18 00:39:11,261 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - B549AB2A9CB84FD99631D30C13718C72: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:11,261 INFO output.MROutput (MROutput.java:close(526)) - A793F7D178534BD5989ACF974AC550B6 closed
2016-05-18 00:39:11,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:11 Completed running task attempt: attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,262 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:11,262 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531950955_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:11,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531950955_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531951255, Process_Duration=6, Process_End_Time=1463531951261, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:11,264 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531950955_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,264 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,264 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:11,264 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F926D9CEA49648F489410244FE47491B, taskAttemptId=attempt_1463531950955_0001_1_01_000000_0, creationTime=1463531951217, allocationTime=1463531951218, startTime=1463531951220, finishTime=1463531951264, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:11,265 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531950955_0001_1_01_000000_0
2016-05-18 00:39:11,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:TASK_FINISHED]: vertexName=F926D9CEA49648F489410244FE47491B, taskId=task_1463531950955_0001_1_01_000000, startTime=1463531951220, finishTime=1463531951265, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463531950955_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531951255, Process_Duration=6, Process_End_Time=1463531951261, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:11,266 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:11,266 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B]
2016-05-18 00:39:11,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:VERTEX_FINISHED]: vertexName=F926D9CEA49648F489410244FE47491B, vertexId=vertex_1463531950955_0001_1_01, initRequestedTime=1463531951125, initedTime=1463531951131, startRequestedTime=1463531951139, startedTime=1463531951140, finishTime=1463531951266, timeTaken=126, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531951255, Process_Duration=6, Process_End_Time=1463531951261, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531951220, firstTasksToStart=[ task_1463531950955_0001_1_01_000000 ], lastTaskFinishTime=1463531951264, lastTasksToFinish=[ task_1463531950955_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531950955_0001_1_01_000000 ], longestDurationTasks=[ task_1463531950955_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:11,267 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:11,267 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531950955_0001_00_000002
2016-05-18 00:39:11,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531950955_0001_00_000002, stoppedTime=1463531951267, exitStatus=0
2016-05-18 00:39:11,267 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:11,268 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,268 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531950955_0001_1_00 [B549AB2A9CB84FD99631D30C13718C72]
2016-05-18 00:39:11,268 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531950955_0001_1
2016-05-18 00:39:11,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531950955_0001_1
2016-05-18 00:39:11,268 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950955_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:11,268 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531950955_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:11,268 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531950955_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:11,268 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: A793F7D178534BD5989ACF974AC550B6 for vertex: vertex_1463531950955_0001_1_01 [F926D9CEA49648F489410244FE47491B], outputName: A793F7D178534BD5989ACF974AC550B6
2016-05-18 00:39:11,269 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531950955_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:11,269 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531950955_0001_00_000002, Exiting
2016-05-18 00:39:11,269 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531950955_0001_00_000002
2016-05-18 00:39:11,269 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531950955_0001_00_000002
2016-05-18 00:39:11,270 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531950955_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:11,270 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531950955_0001_00_000002 completed successfully
2016-05-18 00:39:11,275 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:A793F7D178534BD5989ACF974AC550B6 of vertex/vertexGroup:F926D9CEA49648F489410244FE47491B isVertexGroupOutput:false
2016-05-18 00:39:11,275 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,278 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531950955_0001_1, queueSize=0
2016-05-18 00:39:11,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531950955_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531950955_0001_1, startTime=1463531951123, finishTime=1463531951276, timeTaken=153, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=260, AM_GC_TIME_MILLIS=11, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063902454, Process_Duration=10, Process_End_Time=2927063902464, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:11,278 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531950955_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:11,279 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531950955_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:11 Completed Dag: dag_1463531950955_0001_1
2016-05-18 00:39:11,279 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531950955_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:11,279 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531950955_0001_1_post
2016-05-18 00:39:11,279 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,279 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:11,279 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:11,279 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true, with id=dag_1463531950955_0001_1
2016-05-18 00:39:11,280 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C] comprehensivesort/(1/1) ...sive/true_null_false_true, with id=dag_1463531950955_0001_1
2016-05-18 00:39:11,286 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1463531950955_0001
2016-05-18 00:39:11,287 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:11,287 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:11,287 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:11,288 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:11,288 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:11,288 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:11,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:11,289 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:11,289 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:11,289 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:11,290 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:11,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:11,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:11,290 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:11,294 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/AA7156DFAA22477284D3691111C697FD/B0168BA654AF44BABF0EFDFC9941E57C/.tez/application_1463531950955_0001
2016-05-18 00:39:11,294 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true/_temporary
2016-05-18 00:39:11,295 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.358
2016-05-18 00:39:11,368 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:11,368 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:11,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:39:11,423 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-18 00:39:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:11,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:11,448 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:11,448 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:11,448 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:11,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true
2016-05-18 00:39:11,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:11,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:11,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"]
2016-05-18 00:39:11,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:11,531 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:11,532 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:11,532 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/false_null_true_true
2016-05-18 00:39:11,533 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:11,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:11,550 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:11,551 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:11,551 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:11,559 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001 doesn't exist and is created
2016-05-18 00:39:11,573 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:11,573 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:11,574 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd
2016-05-18 00:39:11,583 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531951550_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:11,583 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:11,584 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:11,585 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:11,585 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:11,585 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:11,585 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:11,586 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:11,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:11,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:11,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:11,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:11,587 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:11,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:11,588 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:11,588 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:11,588 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:11,588 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:11,588 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:11,588 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:11,590 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:11,590 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:11,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:11,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-log-dir
2016-05-18 00:39:11,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-log-dir/history.txt.appattempt_1463531951550_0001_000000, maxErrors=10
2016-05-18 00:39:11,591 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:11,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531951550_0001_000000, appSubmitTime=1463531951583, launchTime=1463531951583
2016-05-18 00:39:11,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:11,598 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59985
2016-05-18 00:39:11,599 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531951550
2016-05-18 00:39:11,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531951550_0001_000000, startTime=1463531951600
2016-05-18 00:39:11,600 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:11,674 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:11,674 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:11,674 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1463531951550_0001, dagName=[FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-05-18 00:39:11,690 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-05-18 00:39:11,691 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:11,692 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531951550_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-log-dir/dag_1463531951550_0001_1.dot
2016-05-18 00:39:11,692 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-log-dir/dag_1463531951550_0001_1-tez-dag.pb.txt
2016-05-18 00:39:11,698 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531951550_0001_1
2016-05-18 00:39:11,699 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,699 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-05-18 00:39:11 Running Dag: dag_1463531951550_0001_1
2016-05-18 00:39:11,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531951550_0001_1, submitTime=1463531951691
2016-05-18 00:39:11,709 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001/tez-conf.pb" } size: 36903 timestamp: 1463531951000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:11,709 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,710 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:11,711 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001/tez-conf.pb" } size: 36903 timestamp: 1463531951000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:11,711 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,711 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:11,711 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:11,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531951550_0001_1, initTime=1463531951713
2016-05-18 00:39:11,713 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531951550_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:11,713 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:11,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:DAG_STARTED]: dagID=dag_1463531951550_0001_1, startTime=1463531951714
2016-05-18 00:39:11,714 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531951550_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A39C8DD2108844189D3B89C8DA0B6117 : {7CB1C89366BD48B69BC3187DEB3D9AB8={InputName=7CB1C89366BD48B69BC3187DEB3D9AB8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 7CB1C89366BD48B69BC3187DEB3D9AB8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,714 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,715 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:11,715 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,718 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 7CB1C89366BD48B69BC3187DEB3D9AB8 on vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,719 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 7CB1C89366BD48B69BC3187DEB3D9AB8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:11,719 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:11,719 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,720 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531951550_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:11,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305], numAdditionalOutputs=1
2016-05-18 00:39:11,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=F5C8AB3CEBDB47B1B0BC30A228405F37, vertex=vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:11,721 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1463531951550_0001, dagName=[FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-05-18 00:39:11,721 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:11,722 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:11,722 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:11,722 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:11,722 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for E57B4549A5404E68817A37AF31EA5305:F5C8AB3CEBDB47B1B0BC30A228405F37 using oldmapred API
2016-05-18 00:39:11,722 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=F5C8AB3CEBDB47B1B0BC30A228405F37, vertexName=E57B4549A5404E68817A37AF31EA5305, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:11,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E57B4549A5404E68817A37AF31EA5305, vertexId=vertex_1463531951550_0001_1_01, initRequestedTime=1463531951715, initedTime=1463531951720, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:11,724 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:11,722 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 7CB1C89366BD48B69BC3187DEB3D9AB8 on vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,725 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:11,725 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {7CB1C89366BD48B69BC3187DEB3D9AB8=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:11,726 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] parallelism set to 1
2016-05-18 00:39:11,726 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:11,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, vertexId=vertex_1463531951550_0001_1_00, initRequestedTime=1463531951714, initedTime=1463531951727, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:11,727 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] with distanceFromRoot: 0
2016-05-18 00:39:11,728 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:11,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531951550_0001_1_00, startRequestedTime=1463531951715, startedTime=1463531951728
2016-05-18 00:39:11,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531951550_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:11,729 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:11,729 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531951550_0001_1_00 for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] numStartedSources: 1 numSources: 1
2016-05-18 00:39:11,729 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] with distanceFromRoot: 1
2016-05-18 00:39:11,729 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A39C8DD2108844189D3B89C8DA0B6117
2016-05-18 00:39:11,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531951550_0001_1_01, startRequestedTime=1463531951729, startedTime=1463531951730
2016-05-18 00:39:11,730 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:11,730 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,731 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: E57B4549A5404E68817A37AF31EA5305 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:11,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_STARTED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, taskId=task_1463531951550_0001_1_00_000000, scheduledTime=1463531951732, launchTime=1463531951732
2016-05-18 00:39:11,732 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,733 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A39C8DD2108844189D3B89C8DA0B6117 in vertex: E57B4549A5404E68817A37AF31EA5305 numBipartiteSourceTasks: 1
2016-05-18 00:39:11,733 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:11,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531951550_0001_00_000001, launchTime=1463531951735
2016-05-18 00:39:11,736 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531951550_0001_00_000001
2016-05-18 00:39:11,737 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531951550_0001_1_00_000000_0] started. Is using containerId: [container_1463531951550_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:11,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0, startTime=1463531951737, containerId=container_1463531951550_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:11,738 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531951550_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11 Starting to run new task attempt: attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,739 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,740 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,740 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:11,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: A39C8DD2108844189D3B89C8DA0B6117, VertexParallelism: 1, TaskAttemptID:attempt_1463531951550_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7CB1C89366BD48B69BC3187DEB3D9AB8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E57B4549A5404E68817A37AF31EA5305, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:11,740 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531951550_0001, with dag id: dag_1463531951550_0001_1
2016-05-18 00:39:11,741 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:11,742 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,744 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531951550_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:11,744 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531951550_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:11,746 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:11,746 INFO input.MRInput (MRInput.java:initialize(440)) - 7CB1C89366BD48B69BC3187DEB3D9AB8 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:11,746 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 7CB1C89366BD48B69BC3187DEB3D9AB8
2016-05-18 00:39:11,752 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A39C8DD2108844189D3B89C8DA0B6117, ordinal: 0
2016-05-18 00:39:11,752 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A39C8DD2108844189D3B89C8DA0B6117, mem on start (mb), free: 86, total: 340, max: 672
2016-05-18 00:39:11,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:11,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:11,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:11,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:11,753 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7CB1C89366BD48B69BC3187DEB3D9AB8:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [E57B4549A5404E68817A37AF31EA5305:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:11,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:11,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:11,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:11,754 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7CB1C89366BD48B69BC3187DEB3D9AB8:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [E57B4549A5404E68817A37AF31EA5305:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 7CB1C89366BD48B69BC3187DEB3D9AB8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 7CB1C89366BD48B69BC3187DEB3D9AB8
2016-05-18 00:39:11,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:11,755 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,756 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:11,756 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 7CB1C89366BD48B69BC3187DEB3D9AB8 initialized RecordReader from event
2016-05-18 00:39:11,757 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:11,758 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 7CB1C89366BD48B69BC3187DEB3D9AB8
2016-05-18 00:39:11,758 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: C22F5EF7DBF04042A446824221612EF4
2016-05-18 00:39:11,758 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] C22F5EF7DBF04042A446824221612EF4
2016-05-18 00:39:11,760 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - E57B4549A5404E68817A37AF31EA5305 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@90f3ce5, comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@4766e308, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:11,760 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for E57B4549A5404E68817A37AF31EA5305: , UsingHashComparator=false
2016-05-18 00:39:11,784 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:11,784 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:11,784 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for E57B4549A5404E68817A37AF31EA5305: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:11,785 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - E57B4549A5404E68817A37AF31EA5305: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:11,785 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:11,785 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A39C8DD2108844189D3B89C8DA0B6117, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:11,789 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,789 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A39C8DD2108844189D3B89C8DA0B6117, mem on close (mb), free: 190, total: 339, max: 672
2016-05-18 00:39:11,790 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - E57B4549A5404E68817A37AF31EA5305: Starting flush of map output
2016-05-18 00:39:11,790 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - E57B4549A5404E68817A37AF31EA5305: Span0.length = 37, perItem = 17
2016-05-18 00:39:11,790 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - E57B4549A5404E68817A37AF31EA5305: done sorting span=0, length=37, time=0
2016-05-18 00:39:11,790 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - E57B4549A5404E68817A37AF31EA5305: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:11,791 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - E57B4549A5404E68817A37AF31EA5305: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-local-dir/output/attempt_1463531951550_0001_1_00_000000_0_10107_0/file.out
2016-05-18 00:39:11,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:11 Completed running task attempt: attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,795 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:11,795 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531951550_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:11,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531951550_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531951785, Process_Duration=4, Process_End_Time=1463531951789, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:11,798 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531951550_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, taskAttemptId=attempt_1463531951550_0001_1_00_000000_0, creationTime=1463531951732, allocationTime=1463531951735, startTime=1463531951737, finishTime=1463531951798, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:11,799 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531951550_0001_1_00_000000_0
2016-05-18 00:39:11,798 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,799 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:11,799 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_FINISHED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, taskId=task_1463531951550_0001_1_00_000000, startTime=1463531951737, finishTime=1463531951799, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1463531951550_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531951785, Process_Duration=4, Process_End_Time=1463531951789, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:11,799 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531951550_0001_00_000001
2016-05-18 00:39:11,800 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:11,800 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_FINISHED]: vertexName=A39C8DD2108844189D3B89C8DA0B6117, vertexId=vertex_1463531951550_0001_1_00, initRequestedTime=1463531951714, initedTime=1463531951727, startRequestedTime=1463531951715, startedTime=1463531951728, finishTime=1463531951800, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531951785, Process_Duration=4, Process_End_Time=1463531951789, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531951737, firstTasksToStart=[ task_1463531951550_0001_1_00_000000 ], lastTaskFinishTime=1463531951798, lastTasksToFinish=[ task_1463531951550_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531951550_0001_1_00_000000 ], longestDurationTasks=[ task_1463531951550_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:11,801 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:11,801 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] attempt: attempt_1463531951550_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:11,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531951550_0001_00_000001, stoppedTime=1463531951802, exitStatus=0
2016-05-18 00:39:11,802 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:11,802 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,802 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E57B4549A5404E68817A37AF31EA5305
2016-05-18 00:39:11,802 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A39C8DD2108844189D3B89C8DA0B6117, destinationVertexName=E57B4549A5404E68817A37AF31EA5305. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:11,802 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,803 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531951550_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:11,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_STARTED]: vertexName=E57B4549A5404E68817A37AF31EA5305, taskId=task_1463531951550_0001_1_01_000000, scheduledTime=1463531951803, launchTime=1463531951803
2016-05-18 00:39:11,804 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,805 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531951550_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:11,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531951550_0001_00_000002, launchTime=1463531951805
2016-05-18 00:39:11,805 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531951550_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:11,806 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531951550_0001_00_000001, Exiting
2016-05-18 00:39:11,806 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531951550_0001_00_000001
2016-05-18 00:39:11,806 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531951550_0001_00_000001
2016-05-18 00:39:11,807 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531951550_0001_00_000002
2016-05-18 00:39:11,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531951550_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:11,811 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531951550_0001_00_000001 completed successfully
2016-05-18 00:39:11,812 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531951550_0001_1_01_000000_0] started. Is using containerId: [container_1463531951550_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:11,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E57B4549A5404E68817A37AF31EA5305, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0, startTime=1463531951812, containerId=container_1463531951550_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:11,813 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531951550_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11 Starting to run new task attempt: attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,813 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,813 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,813 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:11,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: E57B4549A5404E68817A37AF31EA5305, VertexParallelism: 1, TaskAttemptID:attempt_1463531951550_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A39C8DD2108844189D3B89C8DA0B6117, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F5C8AB3CEBDB47B1B0BC30A228405F37, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:11,814 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:11,814 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,816 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531951550_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,816 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531951550_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:11,818 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:11,820 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:11,826 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E57B4549A5404E68817A37AF31EA5305, ordinal: 1
2016-05-18 00:39:11,826 INFO output.MROutput (MROutput.java:initialize(421)) - F5C8AB3CEBDB47B1B0BC30A228405F37: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:11,826 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E57B4549A5404E68817A37AF31EA5305, mem on start (mb), free: 185, total: 339, max: 672
2016-05-18 00:39:11,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:11,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:11,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:11,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:11,827 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[F5C8AB3CEBDB47B1B0BC30A228405F37:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [A39C8DD2108844189D3B89C8DA0B6117:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:11,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:11,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:11,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:11,828 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[F5C8AB3CEBDB47B1B0BC30A228405F37:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [A39C8DD2108844189D3B89C8DA0B6117:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:11,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:11,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A39C8DD2108844189D3B89C8DA0B6117 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:11,829 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A39C8DD2108844189D3B89C8DA0B6117: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:11,829 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A39C8DD2108844189D3B89C8DA0B6117: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:11,830 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A39C8DD2108844189D3B89C8DA0B6117 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:11,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:11,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:11,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A39C8DD2108844189D3B89C8DA0B6117
2016-05-18 00:39:11,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:11,832 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,834 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A39C8DD2108844189D3B89C8DA0B6117: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:11,835 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:11,835 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: C22F5EF7DBF04042A446824221612EF4
2016-05-18 00:39:11,836 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"], id: F5C8AB3CEBDB47B1B0BC30A228405F37
2016-05-18 00:39:11,836 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"]
2016-05-18 00:39:11,836 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] C22F5EF7DBF04042A446824221612EF4, for 1 inputs
2016-05-18 00:39:11,837 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531951550_0001_1_00_000000_0_10107} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531951836, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:11,837 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A39C8DD2108844189D3B89C8DA0B6117
2016-05-18 00:39:11,837 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:11,837 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A39C8DD2108844189D3B89C8DA0B6117, wasInterrupted=false
2016-05-18 00:39:11,839 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:11,839 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:11,840 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001_wd/localmode-local-dir/output/attempt_1463531951550_0001_1_00_000000_0_10107/file.out, len=593, isLocal=true
2016-05-18 00:39:11,840 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:11,840 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:11,840 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:11,841 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:11,841 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A39C8DD2108844189D3B89C8DA0B6117
2016-05-18 00:39:11,841 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E57B4549A5404E68817A37AF31EA5305, all 1 inputs ready in: 00:00:00.005
2016-05-18 00:39:11,841 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A39C8DD2108844189D3B89C8DA0B6117: Shuffle Runner thread complete
2016-05-18 00:39:11,842 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A39C8DD2108844189D3B89C8DA0B6117: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseTupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:11,846 INFO output.MROutput (MROutput.java:close(526)) - F5C8AB3CEBDB47B1B0BC30A228405F37 closed
2016-05-18 00:39:11,846 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531951550_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:11,847 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,847 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E57B4549A5404E68817A37AF31EA5305, mem on close (mb), free: 183, total: 339, max: 672
2016-05-18 00:39:11,847 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A39C8DD2108844189D3B89C8DA0B6117
2016-05-18 00:39:11,847 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A39C8DD2108844189D3B89C8DA0B6117: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:11,847 INFO output.MROutput (MROutput.java:close(526)) - F5C8AB3CEBDB47B1B0BC30A228405F37 closed
2016-05-18 00:39:11,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:11 Completed running task attempt: attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,848 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:11,848 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531951550_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:11,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531951550_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531951841, Process_Duration=6, Process_End_Time=1463531951847, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:11,849 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531951550_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,849 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:11,849 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:11,849 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,850 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531951550_0001_00_000002
2016-05-18 00:39:11,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E57B4549A5404E68817A37AF31EA5305, taskAttemptId=attempt_1463531951550_0001_1_01_000000_0, creationTime=1463531951803, allocationTime=1463531951804, startTime=1463531951812, finishTime=1463531951849, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:11,850 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531951550_0001_1_01_000000_0
2016-05-18 00:39:11,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:TASK_FINISHED]: vertexName=E57B4549A5404E68817A37AF31EA5305, taskId=task_1463531951550_0001_1_01_000000, startTime=1463531951812, finishTime=1463531951851, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463531951550_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531951841, Process_Duration=6, Process_End_Time=1463531951847, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:11,851 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:11,851 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305]
2016-05-18 00:39:11,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:VERTEX_FINISHED]: vertexName=E57B4549A5404E68817A37AF31EA5305, vertexId=vertex_1463531951550_0001_1_01, initRequestedTime=1463531951715, initedTime=1463531951720, startRequestedTime=1463531951729, startedTime=1463531951730, finishTime=1463531951851, timeTaken=121, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531951841, Process_Duration=6, Process_End_Time=1463531951847, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531951812, firstTasksToStart=[ task_1463531951550_0001_1_01_000000 ], lastTaskFinishTime=1463531951849, lastTasksToFinish=[ task_1463531951550_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531951550_0001_1_01_000000 ], longestDurationTasks=[ task_1463531951550_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:11,852 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:11,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531951550_0001_00_000002, stoppedTime=1463531951853, exitStatus=0
2016-05-18 00:39:11,853 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:11,853 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,853 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531951550_0001_1_00 [A39C8DD2108844189D3B89C8DA0B6117]
2016-05-18 00:39:11,853 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531951550_0001_1
2016-05-18 00:39:11,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531951550_0001_1
2016-05-18 00:39:11,853 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: F5C8AB3CEBDB47B1B0BC30A228405F37 for vertex: vertex_1463531951550_0001_1_01 [E57B4549A5404E68817A37AF31EA5305], outputName: F5C8AB3CEBDB47B1B0BC30A228405F37
2016-05-18 00:39:11,855 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531951550_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:11,856 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531951550_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:11,859 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:F5C8AB3CEBDB47B1B0BC30A228405F37 of vertex/vertexGroup:E57B4549A5404E68817A37AF31EA5305 isVertexGroupOutput:false
2016-05-18 00:39:11,860 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:11,861 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531951550_0001_1, queueSize=0
2016-05-18 00:39:11,861 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531951550_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:11,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531951550_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531951550_0001_1, startTime=1463531951713, finishTime=1463531951860, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=270, AM_GC_TIME_MILLIS=11, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063903626, Process_Duration=10, Process_End_Time=2927063903636, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:11,862 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531951550_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:11,862 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531951550_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:11,862 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531951550_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:11 Completed Dag: dag_1463531951550_0001_1
2016-05-18 00:39:11,862 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531951550_0001_00_000002, Exiting
2016-05-18 00:39:11,862 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531951550_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:11,862 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531951550_0001_00_000002
2016-05-18 00:39:11,862 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531951550_0001_00_000002
2016-05-18 00:39:11,862 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531951550_0001_1_post
2016-05-18 00:39:11,862 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531951550_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:11,863 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531951550_0001_00_000002 completed successfully
2016-05-18 00:39:11,863 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:11,863 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:39:11,863 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:11,863 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true, with id=dag_1463531951550_0001_1
2016-05-18 00:39:11,863 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE] comprehensivesort/(1/1) ...sive/false_null_true_true, with id=dag_1463531951550_0001_1
2016-05-18 00:39:11,872 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1463531951550_0001
2016-05-18 00:39:11,873 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:11,873 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:11,873 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:11,874 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:11,874 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:11,875 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:11,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:11,876 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:11,876 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:11,876 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:11,877 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:11,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:11,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:11,879 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:11,882 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/FE04154EC95941C092161DFAF375384E/D859FAB085EE44D9AAA76AADD93364BE/.tez/application_1463531951550_0001
2016-05-18 00:39:11,883 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true/_temporary
2016-05-18 00:39:11,883 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.353
2016-05-18 00:39:11,950 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:11,951 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:11,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 00:39:12,006 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:39:12,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:12,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:12,007 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:12,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:12,029 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true
2016-05-18 00:39:12,072 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:12,072 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:12,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:12,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"]
2016-05-18 00:39:12,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:12,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:12,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:12,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...nsive/true_null_true_true
2016-05-18 00:39:12,085 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:12,098 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:12,102 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:12,103 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:12,103 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:12,112 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001 doesn't exist and is created
2016-05-18 00:39:12,127 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:12,127 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:12,128 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd
2016-05-18 00:39:12,138 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531952103_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:12,138 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:12,138 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:12,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:12,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:12,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:12,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:12,141 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:12,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:12,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:12,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:12,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:12,143 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:12,144 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:12,144 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:12,144 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:12,144 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:12,144 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:12,145 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:12,145 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:12,147 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:12,148 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:12,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:12,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-log-dir
2016-05-18 00:39:12,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-log-dir/history.txt.appattempt_1463531952103_0001_000000, maxErrors=10
2016-05-18 00:39:12,149 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:12,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531952103_0001_000000, appSubmitTime=1463531952137, launchTime=1463531952137
2016-05-18 00:39:12,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:12,155 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34749
2016-05-18 00:39:12,156 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531952103
2016-05-18 00:39:12,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531952103_0001_000000, startTime=1463531952157
2016-05-18 00:39:12,158 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:12,228 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:12,228 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:12,228 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1463531952103_0001, dagName=[5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-05-18 00:39:12,244 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-05-18 00:39:12,245 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:12,245 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531952103_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-log-dir/dag_1463531952103_0001_1.dot
2016-05-18 00:39:12,246 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-log-dir/dag_1463531952103_0001_1-tez-dag.pb.txt
2016-05-18 00:39:12,252 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531952103_0001_1
2016-05-18 00:39:12,252 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,252 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-05-18 00:39:12 Running Dag: dag_1463531952103_0001_1
2016-05-18 00:39:12,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531952103_0001_1, submitTime=1463531952245
2016-05-18 00:39:12,263 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001/tez-conf.pb" } size: 36903 timestamp: 1463531952000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:12,263 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,263 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:12,264 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001/tez-conf.pb" } size: 36903 timestamp: 1463531952000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:12,264 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,264 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:12,265 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:12,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531952103_0001_1, initTime=1463531952265
2016-05-18 00:39:12,266 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952103_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:12,266 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:12,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:DAG_STARTED]: dagID=dag_1463531952103_0001_1, startTime=1463531952266
2016-05-18 00:39:12,266 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1463531952103_0001, dagName=[5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-05-18 00:39:12,266 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952103_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:12,267 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: E222063C72C04CD28075DD01971C0CBA : {62422D1CEC7A4B468A3A00118D73D6EF={InputName=62422D1CEC7A4B468A3A00118D73D6EF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:12,267 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 62422D1CEC7A4B468A3A00118D73D6EF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:12,267 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,267 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,267 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,268 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,268 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:12,268 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,271 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 62422D1CEC7A4B468A3A00118D73D6EF on vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,272 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:12,272 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,272 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531952103_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:12,273 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7], numAdditionalOutputs=1
2016-05-18 00:39:12,273 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=D54903D4F09B493BBE1B038DC08A2E6D, vertex=vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:12,275 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 62422D1CEC7A4B468A3A00118D73D6EF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:12,275 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for DB1F8DEB54D142CD826F38BBBE6B45E7:D54903D4F09B493BBE1B038DC08A2E6D using oldmapred API
2016-05-18 00:39:12,275 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=D54903D4F09B493BBE1B038DC08A2E6D, vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:12,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, vertexId=vertex_1463531952103_0001_1_01, initRequestedTime=1463531952268, initedTime=1463531952273, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:12,277 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:12,277 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:12,277 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:12,278 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:12,278 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:12,278 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 62422D1CEC7A4B468A3A00118D73D6EF on vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,279 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:12,279 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {62422D1CEC7A4B468A3A00118D73D6EF=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:12,280 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] parallelism set to 1
2016-05-18 00:39:12,280 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:12,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E222063C72C04CD28075DD01971C0CBA, vertexId=vertex_1463531952103_0001_1_00, initRequestedTime=1463531952267, initedTime=1463531952280, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:12,281 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] with distanceFromRoot: 0
2016-05-18 00:39:12,281 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:12,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531952103_0001_1_00, startRequestedTime=1463531952268, startedTime=1463531952281
2016-05-18 00:39:12,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531952103_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:12,282 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:12,283 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in E222063C72C04CD28075DD01971C0CBA
2016-05-18 00:39:12,283 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,283 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531952103_0001_1_00 for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] numStartedSources: 1 numSources: 1
2016-05-18 00:39:12,283 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] with distanceFromRoot: 1
2016-05-18 00:39:12,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531952103_0001_1_01, startRequestedTime=1463531952283, startedTime=1463531952284
2016-05-18 00:39:12,285 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:12,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531952103_0001, with dag id: dag_1463531952103_0001_1
2016-05-18 00:39:12,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_STARTED]: vertexName=E222063C72C04CD28075DD01971C0CBA, taskId=task_1463531952103_0001_1_00_000000, scheduledTime=1463531952286, launchTime=1463531952286
2016-05-18 00:39:12,286 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: DB1F8DEB54D142CD826F38BBBE6B45E7 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:12,288 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: E222063C72C04CD28075DD01971C0CBA in vertex: DB1F8DEB54D142CD826F38BBBE6B45E7 numBipartiteSourceTasks: 1
2016-05-18 00:39:12,289 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,290 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:12,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531952103_0001_00_000001, launchTime=1463531952291
2016-05-18 00:39:12,292 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952103_0001_00_000001
2016-05-18 00:39:12,293 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952103_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,293 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531952103_0001_1_00_000000_0] started. Is using containerId: [container_1463531952103_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:12,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E222063C72C04CD28075DD01971C0CBA, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0, startTime=1463531952293, containerId=container_1463531952103_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:12 Starting to run new task attempt: attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,294 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,294 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,294 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:12,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: E222063C72C04CD28075DD01971C0CBA, VertexParallelism: 1, TaskAttemptID:attempt_1463531952103_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=62422D1CEC7A4B468A3A00118D73D6EF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:12,295 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:12,296 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,296 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531952103_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:12,296 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531952103_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:12,300 INFO input.MRInput (MRInput.java:initialize(440)) - 62422D1CEC7A4B468A3A00118D73D6EF using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:12,300 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 62422D1CEC7A4B468A3A00118D73D6EF
2016-05-18 00:39:12,301 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:12,307 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E222063C72C04CD28075DD01971C0CBA, ordinal: 0
2016-05-18 00:39:12,307 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E222063C72C04CD28075DD01971C0CBA, mem on start (mb), free: 85, total: 339, max: 672
2016-05-18 00:39:12,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:12,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:12,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:12,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:12,308 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[62422D1CEC7A4B468A3A00118D73D6EF:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [DB1F8DEB54D142CD826F38BBBE6B45E7:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:12,308 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:12,308 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:12,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:12,309 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[62422D1CEC7A4B468A3A00118D73D6EF:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [DB1F8DEB54D142CD826F38BBBE6B45E7:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:12,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:12,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 62422D1CEC7A4B468A3A00118D73D6EF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:12,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 62422D1CEC7A4B468A3A00118D73D6EF
2016-05-18 00:39:12,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:12,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:12,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:12,310 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,311 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:12,311 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 62422D1CEC7A4B468A3A00118D73D6EF initialized RecordReader from event
2016-05-18 00:39:12,314 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,314 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 62422D1CEC7A4B468A3A00118D73D6EF
2016-05-18 00:39:12,314 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: F0C0AF32C506486292BEFEA2F5073276
2016-05-18 00:39:12,315 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F0C0AF32C506486292BEFEA2F5073276
2016-05-18 00:39:12,317 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - DB1F8DEB54D142CD826F38BBBE6B45E7 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.KeyTuple, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@122779bc, comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@5da92ef3, partitioner=cascading.tuple.tez.util.TuplePartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:12,317 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for DB1F8DEB54D142CD826F38BBBE6B45E7: , UsingHashComparator=false
2016-05-18 00:39:12,343 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:12,344 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:12,344 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for DB1F8DEB54D142CD826F38BBBE6B45E7: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:12,344 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - DB1F8DEB54D142CD826F38BBBE6B45E7: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:12,345 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,346 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E222063C72C04CD28075DD01971C0CBA, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:12,352 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,352 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E222063C72C04CD28075DD01971C0CBA, mem on close (mb), free: 188, total: 338, max: 672
2016-05-18 00:39:12,353 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - DB1F8DEB54D142CD826F38BBBE6B45E7: Starting flush of map output
2016-05-18 00:39:12,353 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - DB1F8DEB54D142CD826F38BBBE6B45E7: Span0.length = 37, perItem = 17
2016-05-18 00:39:12,354 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - DB1F8DEB54D142CD826F38BBBE6B45E7: done sorting span=0, length=37, time=1
2016-05-18 00:39:12,354 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - DB1F8DEB54D142CD826F38BBBE6B45E7: Heap = SpanIterator<0:36> (span=Span[16777216,629]),
2016-05-18 00:39:12,355 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - DB1F8DEB54D142CD826F38BBBE6B45E7: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-local-dir/output/attempt_1463531952103_0001_1_00_000000_0_10113_0/file.out
2016-05-18 00:39:12,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:12 Completed running task attempt: attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,359 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:12,360 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531952103_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:12,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531952103_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531952346, Process_Duration=6, Process_End_Time=1463531952352, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:12,361 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531952103_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E222063C72C04CD28075DD01971C0CBA, taskAttemptId=attempt_1463531952103_0001_1_00_000000_0, creationTime=1463531952286, allocationTime=1463531952291, startTime=1463531952293, finishTime=1463531952361, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:12,362 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,362 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531952103_0001_1_00_000000_0
2016-05-18 00:39:12,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_FINISHED]: vertexName=E222063C72C04CD28075DD01971C0CBA, taskId=task_1463531952103_0001_1_00_000000, startTime=1463531952293, finishTime=1463531952362, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1463531952103_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531952346, Process_Duration=6, Process_End_Time=1463531952352, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:12,363 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:12,363 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,363 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:12,363 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,364 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952103_0001_00_000001
2016-05-18 00:39:12,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_FINISHED]: vertexName=E222063C72C04CD28075DD01971C0CBA, vertexId=vertex_1463531952103_0001_1_00, initRequestedTime=1463531952267, initedTime=1463531952280, startRequestedTime=1463531952268, startedTime=1463531952281, finishTime=1463531952363, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531952346, Process_Duration=6, Process_End_Time=1463531952352, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531952293, firstTasksToStart=[ task_1463531952103_0001_1_00_000000 ], lastTaskFinishTime=1463531952361, lastTasksToFinish=[ task_1463531952103_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531952103_0001_1_00_000000 ], longestDurationTasks=[ task_1463531952103_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:12,364 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:12,364 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] attempt: attempt_1463531952103_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:12,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531952103_0001_00_000001, stoppedTime=1463531952365, exitStatus=0
2016-05-18 00:39:12,365 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:12,365 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DB1F8DEB54D142CD826F38BBBE6B45E7
2016-05-18 00:39:12,365 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,365 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=E222063C72C04CD28075DD01971C0CBA, destinationVertexName=DB1F8DEB54D142CD826F38BBBE6B45E7. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:12,365 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531952103_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:12,365 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_STARTED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, taskId=task_1463531952103_0001_1_01_000000, scheduledTime=1463531952366, launchTime=1463531952366
2016-05-18 00:39:12,365 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531952103_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:12,367 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,367 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952103_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:12,368 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531952103_0001_00_000001, Exiting
2016-05-18 00:39:12,368 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531952103_0001_00_000001
2016-05-18 00:39:12,368 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531952103_0001_00_000001
2016-05-18 00:39:12,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531952103_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:12,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531952103_0001_00_000001 completed successfully
2016-05-18 00:39:12,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531952103_0001_00_000002, launchTime=1463531952371
2016-05-18 00:39:12,373 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952103_0001_00_000002
2016-05-18 00:39:12,373 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531952103_0001_1_01_000000_0] started. Is using containerId: [container_1463531952103_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:12,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0, startTime=1463531952373, containerId=container_1463531952103_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:12,374 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952103_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12 Starting to run new task attempt: attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,375 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,376 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,376 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:12,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: DB1F8DEB54D142CD826F38BBBE6B45E7, VertexParallelism: 1, TaskAttemptID:attempt_1463531952103_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E222063C72C04CD28075DD01971C0CBA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D54903D4F09B493BBE1B038DC08A2E6D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:12,376 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:12,377 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,378 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531952103_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,378 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531952103_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:12,383 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:12,390 INFO output.MROutput (MROutput.java:initialize(421)) - D54903D4F09B493BBE1B038DC08A2E6D: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:12,391 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:12,392 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DB1F8DEB54D142CD826F38BBBE6B45E7, ordinal: 1
2016-05-18 00:39:12,393 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB1F8DEB54D142CD826F38BBBE6B45E7, mem on start (mb), free: 182, total: 338, max: 672
2016-05-18 00:39:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:12,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:12,394 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[D54903D4F09B493BBE1B038DC08A2E6D:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [E222063C72C04CD28075DD01971C0CBA:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:12,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:12,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:12,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:12,395 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[D54903D4F09B493BBE1B038DC08A2E6D:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [E222063C72C04CD28075DD01971C0CBA:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:12,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:12,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: E222063C72C04CD28075DD01971C0CBA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:12,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:12,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:12,398 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - E222063C72C04CD28075DD01971C0CBA: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:12,398 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - E222063C72C04CD28075DD01971C0CBA: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:12,399 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: E222063C72C04CD28075DD01971C0CBA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:12,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: E222063C72C04CD28075DD01971C0CBA
2016-05-18 00:39:12,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:12,402 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,403 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E222063C72C04CD28075DD01971C0CBA: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:12,405 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531952103_0001_1_00_000000_0_10113} to DISK_DIRECT, csize=593, dsize=589, EndTime=1463531952405, TimeTaken=1, Rate=0.57 MB/s
2016-05-18 00:39:12,405 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : E222063C72C04CD28075DD01971C0CBA
2016-05-18 00:39:12,405 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:12,408 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: E222063C72C04CD28075DD01971C0CBA, wasInterrupted=false
2016-05-18 00:39:12,408 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:12,410 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:12,410 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:12,410 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001_wd/localmode-local-dir/output/attempt_1463531952103_0001_1_00_000000_0_10113/file.out, len=593, isLocal=true
2016-05-18 00:39:12,411 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 593 bytes from disk
2016-05-18 00:39:12,411 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:12,411 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:12,412 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-05-18 00:39:12,412 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : E222063C72C04CD28075DD01971C0CBA
2016-05-18 00:39:12,412 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - E222063C72C04CD28075DD01971C0CBA: Shuffle Runner thread complete
2016-05-18 00:39:12,412 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: F0C0AF32C506486292BEFEA2F5073276
2016-05-18 00:39:12,412 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"], id: D54903D4F09B493BBE1B038DC08A2E6D
2016-05-18 00:39:12,413 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"]
2016-05-18 00:39:12,413 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F0C0AF32C506486292BEFEA2F5073276, for 1 inputs
2016-05-18 00:39:12,413 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DB1F8DEB54D142CD826F38BBBE6B45E7, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:12,414 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - E222063C72C04CD28075DD01971C0CBA: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseTupleComparator, keyClass=cascading.tuple.io.KeyTuple, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:12,421 INFO output.MROutput (MROutput.java:close(526)) - D54903D4F09B493BBE1B038DC08A2E6D closed
2016-05-18 00:39:12,423 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531952103_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:12,423 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,424 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DB1F8DEB54D142CD826F38BBBE6B45E7, mem on close (mb), free: 180, total: 338, max: 672
2016-05-18 00:39:12,424 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: E222063C72C04CD28075DD01971C0CBA
2016-05-18 00:39:12,424 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - E222063C72C04CD28075DD01971C0CBA: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:12,424 INFO output.MROutput (MROutput.java:close(526)) - D54903D4F09B493BBE1B038DC08A2E6D closed
2016-05-18 00:39:12,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:12 Completed running task attempt: attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,425 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:12,425 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531952103_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:12,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531952103_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531952413, Process_Duration=10, Process_End_Time=1463531952423, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:12,427 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531952103_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,427 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,427 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:12,427 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, taskAttemptId=attempt_1463531952103_0001_1_01_000000_0, creationTime=1463531952367, allocationTime=1463531952368, startTime=1463531952373, finishTime=1463531952427, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:12,428 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531952103_0001_1_01_000000_0
2016-05-18 00:39:12,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:TASK_FINISHED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, taskId=task_1463531952103_0001_1_01_000000, startTime=1463531952373, finishTime=1463531952428, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463531952103_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531952413, Process_Duration=10, Process_End_Time=1463531952423, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:12,429 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:12,429 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7]
2016-05-18 00:39:12,429 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952103_0001_00_000002
2016-05-18 00:39:12,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:VERTEX_FINISHED]: vertexName=DB1F8DEB54D142CD826F38BBBE6B45E7, vertexId=vertex_1463531952103_0001_1_01, initRequestedTime=1463531952268, initedTime=1463531952273, startRequestedTime=1463531952283, startedTime=1463531952284, finishTime=1463531952429, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531952413, Process_Duration=10, Process_End_Time=1463531952423, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531952373, firstTasksToStart=[ task_1463531952103_0001_1_01_000000 ], lastTaskFinishTime=1463531952427, lastTasksToFinish=[ task_1463531952103_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531952103_0001_1_01_000000 ], longestDurationTasks=[ task_1463531952103_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:12,430 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:12,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531952103_0001_00_000002, stoppedTime=1463531952430, exitStatus=0
2016-05-18 00:39:12,430 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:12,430 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,431 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531952103_0001_1_00 [E222063C72C04CD28075DD01971C0CBA]
2016-05-18 00:39:12,431 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531952103_0001_1
2016-05-18 00:39:12,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531952103_0001_1
2016-05-18 00:39:12,431 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952103_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:12,431 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531952103_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:12,431 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: D54903D4F09B493BBE1B038DC08A2E6D for vertex: vertex_1463531952103_0001_1_01 [DB1F8DEB54D142CD826F38BBBE6B45E7], outputName: D54903D4F09B493BBE1B038DC08A2E6D
2016-05-18 00:39:12,431 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531952103_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:12,431 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952103_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:12,431 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531952103_0001_00_000002, Exiting
2016-05-18 00:39:12,431 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531952103_0001_00_000002
2016-05-18 00:39:12,432 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531952103_0001_00_000002
2016-05-18 00:39:12,432 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531952103_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:12,432 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531952103_0001_00_000002 completed successfully
2016-05-18 00:39:12,438 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:D54903D4F09B493BBE1B038DC08A2E6D of vertex/vertexGroup:DB1F8DEB54D142CD826F38BBBE6B45E7 isVertexGroupOutput:false
2016-05-18 00:39:12,438 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,439 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531952103_0001_1, queueSize=0
2016-05-18 00:39:12,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952103_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531952103_0001_1, startTime=1463531952266, finishTime=1463531952438, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=370, AM_GC_TIME_MILLIS=12, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=5, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063904759, Process_Duration=16, Process_End_Time=2927063904775, Read_Duration=6, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:12,440 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531952103_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:12,440 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952103_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:12 Completed Dag: dag_1463531952103_0001_1
2016-05-18 00:39:12,440 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531952103_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:12,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531952103_0001_1_post
2016-05-18 00:39:12,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true, with id=dag_1463531952103_0001_1
2016-05-18 00:39:12,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC] comprehensivesort/(1/1) ...nsive/true_null_true_true, with id=dag_1463531952103_0001_1
2016-05-18 00:39:12,450 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1463531952103_0001
2016-05-18 00:39:12,452 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:12,452 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:12,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:12,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:12,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:12,453 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:12,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:12,453 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:12,454 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:12,454 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:12,454 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:12,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:12,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:12,455 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:12,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/5D8AD2E86042476F862CCF5F49576F4A/F7B17B39A9954519A9D4B0E7AAE76BBC/.tez/application_1463531952103_0001
2016-05-18 00:39:12,459 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true/_temporary
2016-05-18 00:39:12,459 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.378
2016-05-18 00:39:12,528 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:12,528 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:12,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-18 00:39:12,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-18 00:39:12,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:12,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:12,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:12,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:12,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true
2016-05-18 00:39:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"]
2016-05-18 00:39:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:12,654 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:12,654 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_false_true
2016-05-18 00:39:12,654 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:12,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:12,671 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:12,671 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:12,672 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:12,680 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001 doesn't exist and is created
2016-05-18 00:39:12,695 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:12,695 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:12,696 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd
2016-05-18 00:39:12,705 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531952671_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:12,706 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:12,706 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:12,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:12,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:12,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:12,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:12,708 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:12,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:12,709 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:12,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:12,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:12,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:12,710 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:12,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:12,710 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:12,711 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:12,711 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:12,711 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:12,711 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:12,713 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:12,714 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:12,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:12,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-log-dir
2016-05-18 00:39:12,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-log-dir/history.txt.appattempt_1463531952671_0001_000000, maxErrors=10
2016-05-18 00:39:12,715 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:12,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531952671_0001_000000, appSubmitTime=1463531952705, launchTime=1463531952705
2016-05-18 00:39:12,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:12,720 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:12,720 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:12,720 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34183
2016-05-18 00:39:12,721 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531952671
2016-05-18 00:39:12,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531952671_0001_000000, startTime=1463531952723
2016-05-18 00:39:12,724 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:12,796 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:12,796 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:12,796 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1463531952671_0001, dagName=[D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-05-18 00:39:12,812 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-05-18 00:39:12,812 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:12,813 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531952671_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-log-dir/dag_1463531952671_0001_1.dot
2016-05-18 00:39:12,813 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-log-dir/dag_1463531952671_0001_1-tez-dag.pb.txt
2016-05-18 00:39:12,819 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531952671_0001_1
2016-05-18 00:39:12,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,819 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-05-18 00:39:12 Running Dag: dag_1463531952671_0001_1
2016-05-18 00:39:12,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531952671_0001_1, submitTime=1463531952812
2016-05-18 00:39:12,828 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001/tez-conf.pb" } size: 36903 timestamp: 1463531952000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:12,829 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,829 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:12,831 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001/tez-conf.pb" } size: 36903 timestamp: 1463531952000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:12,831 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,831 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:12,831 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:12,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531952671_0001_1, initTime=1463531952833
2016-05-18 00:39:12,833 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952671_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:12,834 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:12,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:DAG_STARTED]: dagID=dag_1463531952671_0001_1, startTime=1463531952834
2016-05-18 00:39:12,834 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952671_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:12,834 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: DF04981AB2E349F48E6AE262D5444D28 : {FEAE817F84D84A1890C6C64B9C90F939={InputName=FEAE817F84D84A1890C6C64B9C90F939}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:12,834 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: FEAE817F84D84A1890C6C64B9C90F939, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:12,835 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,835 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1463531952671_0001, dagName=[D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-05-18 00:39:12,835 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,835 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,835 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,836 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:12,836 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: FEAE817F84D84A1890C6C64B9C90F939 on vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,836 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,838 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input FEAE817F84D84A1890C6C64B9C90F939 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:12,839 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:12,840 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,839 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:12,840 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,840 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:12,840 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:12,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531952671_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:12,841 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE], numAdditionalOutputs=1
2016-05-18 00:39:12,841 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=25A5838317424D22A6233B77A84D081F, vertex=vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:12,841 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:12,843 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: FEAE817F84D84A1890C6C64B9C90F939 on vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,843 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D580F0CFE13F444AA837772BB996E2FE:25A5838317424D22A6233B77A84D081F using oldmapred API
2016-05-18 00:39:12,843 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=25A5838317424D22A6233B77A84D081F, vertexName=D580F0CFE13F444AA837772BB996E2FE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:12,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, vertexId=vertex_1463531952671_0001_1_01, initRequestedTime=1463531952836, initedTime=1463531952841, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:12,844 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:12,847 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:12,847 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {FEAE817F84D84A1890C6C64B9C90F939=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:12,848 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] parallelism set to 1
2016-05-18 00:39:12,848 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:12,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, vertexId=vertex_1463531952671_0001_1_00, initRequestedTime=1463531952834, initedTime=1463531952849, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:12,849 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] with distanceFromRoot: 0
2016-05-18 00:39:12,849 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:12,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531952671_0001_1_00, startRequestedTime=1463531952836, startedTime=1463531952849
2016-05-18 00:39:12,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531952671_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:12,850 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:12,850 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531952671_0001_1_00 for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] numStartedSources: 1 numSources: 1
2016-05-18 00:39:12,851 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] with distanceFromRoot: 1
2016-05-18 00:39:12,850 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in DF04981AB2E349F48E6AE262D5444D28
2016-05-18 00:39:12,851 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531952671_0001_1_01, startRequestedTime=1463531952850, startedTime=1463531952851
2016-05-18 00:39:12,852 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:12,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_STARTED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, taskId=task_1463531952671_0001_1_00_000000, scheduledTime=1463531952852, launchTime=1463531952852
2016-05-18 00:39:12,853 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,853 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D580F0CFE13F444AA837772BB996E2FE with 0 source tasks and 1 pending tasks
2016-05-18 00:39:12,854 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: DF04981AB2E349F48E6AE262D5444D28 in vertex: D580F0CFE13F444AA837772BB996E2FE numBipartiteSourceTasks: 1
2016-05-18 00:39:12,854 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:12,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531952671_0001_00_000001, launchTime=1463531952855
2016-05-18 00:39:12,856 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952671_0001_00_000001
2016-05-18 00:39:12,857 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531952671_0001_1_00_000000_0] started. Is using containerId: [container_1463531952671_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:12,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0, startTime=1463531952857, containerId=container_1463531952671_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:12,858 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952671_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12 Starting to run new task attempt: attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,859 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,859 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,859 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:12,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: DF04981AB2E349F48E6AE262D5444D28, VertexParallelism: 1, TaskAttemptID:attempt_1463531952671_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FEAE817F84D84A1890C6C64B9C90F939, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D580F0CFE13F444AA837772BB996E2FE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:12,860 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:12,861 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531952671_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:12,861 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531952671_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:12,861 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531952671_0001, with dag id: dag_1463531952671_0001_1
2016-05-18 00:39:12,865 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:12,867 INFO input.MRInput (MRInput.java:initialize(440)) - FEAE817F84D84A1890C6C64B9C90F939 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:12,867 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: FEAE817F84D84A1890C6C64B9C90F939
2016-05-18 00:39:12,871 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF04981AB2E349F48E6AE262D5444D28, ordinal: 0
2016-05-18 00:39:12,872 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF04981AB2E349F48E6AE262D5444D28, mem on start (mb), free: 82, total: 338, max: 672
2016-05-18 00:39:12,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:12,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:12,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:12,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:12,873 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[FEAE817F84D84A1890C6C64B9C90F939:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D580F0CFE13F444AA837772BB996E2FE:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:12,873 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:12,873 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:12,873 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:12,873 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[FEAE817F84D84A1890C6C64B9C90F939:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D580F0CFE13F444AA837772BB996E2FE:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: FEAE817F84D84A1890C6C64B9C90F939 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: FEAE817F84D84A1890C6C64B9C90F939
2016-05-18 00:39:12,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:12,875 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,877 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:12,877 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - FEAE817F84D84A1890C6C64B9C90F939 initialized RecordReader from event
2016-05-18 00:39:12,878 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,879 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: FEAE817F84D84A1890C6C64B9C90F939
2016-05-18 00:39:12,879 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 4EDF64E555A94CB0B632A921E3205392
2016-05-18 00:39:12,879 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 4EDF64E555A94CB0B632A921E3205392
2016-05-18 00:39:12,881 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D580F0CFE13F444AA837772BB996E2FE using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@7ab01c86, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@4020e057, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:12,881 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D580F0CFE13F444AA837772BB996E2FE: , UsingHashComparator=false
2016-05-18 00:39:12,911 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:12,911 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:12,911 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D580F0CFE13F444AA837772BB996E2FE: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:12,911 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D580F0CFE13F444AA837772BB996E2FE: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:12,912 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:12,912 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF04981AB2E349F48E6AE262D5444D28, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:12,916 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,917 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF04981AB2E349F48E6AE262D5444D28, mem on close (mb), free: 183, total: 335, max: 672
2016-05-18 00:39:12,917 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D580F0CFE13F444AA837772BB996E2FE: Starting flush of map output
2016-05-18 00:39:12,917 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D580F0CFE13F444AA837772BB996E2FE: Span0.length = 37, perItem = 24
2016-05-18 00:39:12,918 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D580F0CFE13F444AA837772BB996E2FE: done sorting span=0, length=37, time=0
2016-05-18 00:39:12,919 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D580F0CFE13F444AA837772BB996E2FE: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:12,920 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D580F0CFE13F444AA837772BB996E2FE: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-local-dir/output/attempt_1463531952671_0001_1_00_000000_0_10118_0/file.out
2016-05-18 00:39:12,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:12 Completed running task attempt: attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,924 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:12,924 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531952671_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:12,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531952671_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531952912, Process_Duration=4, Process_End_Time=1463531952916, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:12,926 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531952671_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, taskAttemptId=attempt_1463531952671_0001_1_00_000000_0, creationTime=1463531952853, allocationTime=1463531952855, startTime=1463531952857, finishTime=1463531952926, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:12,926 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,926 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:12,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,926 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531952671_0001_1_00_000000_0
2016-05-18 00:39:12,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_FINISHED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, taskId=task_1463531952671_0001_1_00_000000, startTime=1463531952857, finishTime=1463531952927, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1463531952671_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531952912, Process_Duration=4, Process_End_Time=1463531952916, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:12,928 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952671_0001_00_000001
2016-05-18 00:39:12,928 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:12,928 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_FINISHED]: vertexName=DF04981AB2E349F48E6AE262D5444D28, vertexId=vertex_1463531952671_0001_1_00, initRequestedTime=1463531952834, initedTime=1463531952849, startRequestedTime=1463531952836, startedTime=1463531952849, finishTime=1463531952928, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531952912, Process_Duration=4, Process_End_Time=1463531952916, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531952857, firstTasksToStart=[ task_1463531952671_0001_1_00_000000 ], lastTaskFinishTime=1463531952926, lastTasksToFinish=[ task_1463531952671_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531952671_0001_1_00_000000 ], longestDurationTasks=[ task_1463531952671_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:12,929 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:12,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531952671_0001_00_000001, stoppedTime=1463531952929, exitStatus=0
2016-05-18 00:39:12,930 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] attempt: attempt_1463531952671_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:12,930 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D580F0CFE13F444AA837772BB996E2FE
2016-05-18 00:39:12,930 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=DF04981AB2E349F48E6AE262D5444D28, destinationVertexName=D580F0CFE13F444AA837772BB996E2FE. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:12,930 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,931 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:12,931 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,931 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531952671_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:12,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_STARTED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, taskId=task_1463531952671_0001_1_01_000000, scheduledTime=1463531952931, launchTime=1463531952931
2016-05-18 00:39:12,931 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531952671_0001_00_000002, launchTime=1463531952933
2016-05-18 00:39:12,933 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952671_0001_00_000002
2016-05-18 00:39:12,934 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952671_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,934 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531952671_0001_1_01_000000_0] started. Is using containerId: [container_1463531952671_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:12,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0, startTime=1463531952934, containerId=container_1463531952671_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:12 Starting to run new task attempt: attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,935 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,935 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,935 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:12,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: D580F0CFE13F444AA837772BB996E2FE, VertexParallelism: 1, TaskAttemptID:attempt_1463531952671_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DF04981AB2E349F48E6AE262D5444D28, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=25A5838317424D22A6233B77A84D081F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:12,936 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:12,936 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,937 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531952671_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,937 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531952671_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:12,938 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531952671_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:12,938 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952671_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:12,939 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531952671_0001_00_000001, Exiting
2016-05-18 00:39:12,939 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531952671_0001_00_000001
2016-05-18 00:39:12,939 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531952671_0001_00_000001
2016-05-18 00:39:12,940 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531952671_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:12,940 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531952671_0001_00_000001 completed successfully
2016-05-18 00:39:12,940 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:12,941 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:12,945 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D580F0CFE13F444AA837772BB996E2FE, ordinal: 1
2016-05-18 00:39:12,945 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D580F0CFE13F444AA837772BB996E2FE, mem on start (mb), free: 179, total: 335, max: 672
2016-05-18 00:39:12,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:12,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:12,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:12,953 INFO output.MROutput (MROutput.java:initialize(421)) - 25A5838317424D22A6233B77A84D081F: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:12,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:12,953 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[DF04981AB2E349F48E6AE262D5444D28:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [25A5838317424D22A6233B77A84D081F:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:12,953 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:12,953 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:12,954 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:12,954 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[DF04981AB2E349F48E6AE262D5444D28:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [25A5838317424D22A6233B77A84D081F:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:12,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:12,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: DF04981AB2E349F48E6AE262D5444D28 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:12,954 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - DF04981AB2E349F48E6AE262D5444D28: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:12,954 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - DF04981AB2E349F48E6AE262D5444D28: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:12,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:12,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:12,955 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: DF04981AB2E349F48E6AE262D5444D28 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:12,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: DF04981AB2E349F48E6AE262D5444D28
2016-05-18 00:39:12,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:12,958 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,960 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - DF04981AB2E349F48E6AE262D5444D28: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:12,962 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531952671_0001_1_00_000000_0_10118} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531952962, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:12,962 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : DF04981AB2E349F48E6AE262D5444D28
2016-05-18 00:39:12,962 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:12,963 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: DF04981AB2E349F48E6AE262D5444D28, wasInterrupted=false
2016-05-18 00:39:12,963 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:12,963 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:12,964 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:12,964 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 4EDF64E555A94CB0B632A921E3205392
2016-05-18 00:39:12,964 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"], id: 25A5838317424D22A6233B77A84D081F
2016-05-18 00:39:12,964 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001_wd/localmode-local-dir/output/attempt_1463531952671_0001_1_00_000000_0_10118/file.out, len=732, isLocal=true
2016-05-18 00:39:12,964 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"]
2016-05-18 00:39:12,964 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:12,964 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:12,965 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:12,965 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - DF04981AB2E349F48E6AE262D5444D28: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:12,965 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:12,965 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : DF04981AB2E349F48E6AE262D5444D28
2016-05-18 00:39:12,965 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - DF04981AB2E349F48E6AE262D5444D28: Shuffle Runner thread complete
2016-05-18 00:39:12,965 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 4EDF64E555A94CB0B632A921E3205392, for 1 inputs
2016-05-18 00:39:12,966 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D580F0CFE13F444AA837772BB996E2FE, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:12,966 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - DF04981AB2E349F48E6AE262D5444D28: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:12,975 INFO output.MROutput (MROutput.java:close(526)) - 25A5838317424D22A6233B77A84D081F closed
2016-05-18 00:39:12,975 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531952671_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:12,975 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,976 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D580F0CFE13F444AA837772BB996E2FE, mem on close (mb), free: 175, total: 335, max: 672
2016-05-18 00:39:12,976 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: DF04981AB2E349F48E6AE262D5444D28
2016-05-18 00:39:12,976 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - DF04981AB2E349F48E6AE262D5444D28: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:12,976 INFO output.MROutput (MROutput.java:close(526)) - 25A5838317424D22A6233B77A84D081F closed
2016-05-18 00:39:12,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:12 Completed running task attempt: attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,977 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:12,977 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531952671_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:12,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531952671_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531952966, Process_Duration=9, Process_End_Time=1463531952975, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:12,979 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531952671_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, taskAttemptId=attempt_1463531952671_0001_1_01_000000_0, creationTime=1463531952931, allocationTime=1463531952932, startTime=1463531952934, finishTime=1463531952979, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:12,979 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:12,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:12,979 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531952671_0001_1_01_000000_0
2016-05-18 00:39:12,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:TASK_FINISHED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, taskId=task_1463531952671_0001_1_01_000000, startTime=1463531952934, finishTime=1463531952979, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463531952671_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531952966, Process_Duration=9, Process_End_Time=1463531952975, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:12,980 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:12,980 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE]
2016-05-18 00:39:12,980 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:VERTEX_FINISHED]: vertexName=D580F0CFE13F444AA837772BB996E2FE, vertexId=vertex_1463531952671_0001_1_01, initRequestedTime=1463531952836, initedTime=1463531952841, startRequestedTime=1463531952850, startedTime=1463531952851, finishTime=1463531952980, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531952966, Process_Duration=9, Process_End_Time=1463531952975, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531952934, firstTasksToStart=[ task_1463531952671_0001_1_01_000000 ], lastTaskFinishTime=1463531952979, lastTasksToFinish=[ task_1463531952671_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531952671_0001_1_01_000000 ], longestDurationTasks=[ task_1463531952671_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:12,981 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:12,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531952671_0001_00_000002, stoppedTime=1463531952981, exitStatus=0
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531952671_0001_1_00 [DF04981AB2E349F48E6AE262D5444D28]
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531952671_0001_1
2016-05-18 00:39:12,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531952671_0001_1
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 25A5838317424D22A6233B77A84D081F for vertex: vertex_1463531952671_0001_1_01 [D580F0CFE13F444AA837772BB996E2FE], outputName: 25A5838317424D22A6233B77A84D081F
2016-05-18 00:39:12,981 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531952671_0001_00_000002
2016-05-18 00:39:12,982 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952671_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:12,988 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531952671_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:12,988 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531952671_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:12,988 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531952671_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:12,989 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531952671_0001_00_000002, Exiting
2016-05-18 00:39:12,989 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531952671_0001_00_000002
2016-05-18 00:39:12,989 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531952671_0001_00_000002
2016-05-18 00:39:12,989 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531952671_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:12,989 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531952671_0001_00_000002 completed successfully
2016-05-18 00:39:12,992 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:25A5838317424D22A6233B77A84D081F of vertex/vertexGroup:D580F0CFE13F444AA837772BB996E2FE isVertexGroupOutput:false
2016-05-18 00:39:12,992 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:12,993 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531952671_0001_1, queueSize=0
2016-05-18 00:39:12,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531952671_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531952671_0001_1, startTime=1463531952834, finishTime=1463531952992, timeTaken=158, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=300, AM_GC_TIME_MILLIS=16, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063905878, Process_Duration=13, Process_End_Time=2927063905891, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:12,994 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531952671_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:12,994 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531952671_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:12 Completed Dag: dag_1463531952671_0001_1
2016-05-18 00:39:12,994 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531952671_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:12,994 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531952671_0001_1_post
2016-05-18 00:39:12,994 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:12,994 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:12,994 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:12,994 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true, with id=dag_1463531952671_0001_1
2016-05-18 00:39:12,994 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A] comprehensivesort/(1/1) ...ve/false_false_false_true, with id=dag_1463531952671_0001_1
2016-05-18 00:39:13,005 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1463531952671_0001
2016-05-18 00:39:13,007 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:13,007 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:13,007 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:13,007 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:13,007 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:13,007 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:13,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:13,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:13,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:13,011 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:13,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:13,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:13,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:13,012 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:13,016 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/D3D2414C1AA244CE8C7C07A60FF41352/B73E2731416E499785459C6D0F71283A/.tez/application_1463531952671_0001
2016-05-18 00:39:13,017 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true/_temporary
2016-05-18 00:39:13,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.364
2016-05-18 00:39:13,100 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:13,101 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:13,151 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-18 00:39:13,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-18 00:39:13,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:13,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:13,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:13,173 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:13,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true
2016-05-18 00:39:13,226 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"]
2016-05-18 00:39:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:13,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:13,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_false_true
2016-05-18 00:39:13,228 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:13,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:13,246 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:13,246 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:13,246 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:13,255 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001 doesn't exist and is created
2016-05-18 00:39:13,269 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:13,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:13,270 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd
2016-05-18 00:39:13,281 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531953246_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:13,281 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:13,281 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:13,283 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:13,283 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:13,284 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:13,284 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:13,284 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:13,286 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:13,286 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:13,287 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:13,287 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:13,287 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:13,287 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:13,287 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:13,288 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:13,288 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:13,288 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:13,288 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:13,289 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:13,291 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:13,292 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:13,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:13,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-log-dir
2016-05-18 00:39:13,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-log-dir/history.txt.appattempt_1463531953246_0001_000000, maxErrors=10
2016-05-18 00:39:13,294 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:13,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531953246_0001_000000, appSubmitTime=1463531953280, launchTime=1463531953281
2016-05-18 00:39:13,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:13,300 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42801
2016-05-18 00:39:13,302 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531953246
2016-05-18 00:39:13,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531953246_0001_000000, startTime=1463531953304
2016-05-18 00:39:13,304 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:13,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:13,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:13,370 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:13,370 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:13,370 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1463531953246_0001, dagName=[BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-05-18 00:39:13,386 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-05-18 00:39:13,387 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:13,387 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531953246_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-log-dir/dag_1463531953246_0001_1.dot
2016-05-18 00:39:13,388 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-log-dir/dag_1463531953246_0001_1-tez-dag.pb.txt
2016-05-18 00:39:13,394 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531953246_0001_1
2016-05-18 00:39:13,394 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,394 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-05-18 00:39:13 Running Dag: dag_1463531953246_0001_1
2016-05-18 00:39:13,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531953246_0001_1, submitTime=1463531953387
2016-05-18 00:39:13,404 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001/tez-conf.pb" } size: 36903 timestamp: 1463531953000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:13,404 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,404 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:13,405 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001/tez-conf.pb" } size: 36903 timestamp: 1463531953000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:13,405 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,406 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:13,406 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:13,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531953246_0001_1, initTime=1463531953406
2016-05-18 00:39:13,407 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953246_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:13,407 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:13,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:DAG_STARTED]: dagID=dag_1463531953246_0001_1, startTime=1463531953407
2016-05-18 00:39:13,408 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953246_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D1C857C118D64F478D7F3881E3121865 : {7D82DFADB9454D12B60E57CCFD3BA219={InputName=7D82DFADB9454D12B60E57CCFD3BA219}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 7D82DFADB9454D12B60E57CCFD3BA219, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,408 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,409 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:13,409 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,409 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1463531953246_0001, dagName=[BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-05-18 00:39:13,411 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 7D82DFADB9454D12B60E57CCFD3BA219 on vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,412 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:13,412 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,412 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531953246_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:13,413 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2], numAdditionalOutputs=1
2016-05-18 00:39:13,413 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=3B37CEDA695244A1A886832A0F40EBA1, vertex=vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:13,415 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 7D82DFADB9454D12B60E57CCFD3BA219 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:13,415 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D30F424DD9A8406F980F5352577249C2:3B37CEDA695244A1A886832A0F40EBA1 using oldmapred API
2016-05-18 00:39:13,415 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=3B37CEDA695244A1A886832A0F40EBA1, vertexName=D30F424DD9A8406F980F5352577249C2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:13,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D30F424DD9A8406F980F5352577249C2, vertexId=vertex_1463531953246_0001_1_01, initRequestedTime=1463531953409, initedTime=1463531953413, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:13,417 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:13,417 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:13,417 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:13,417 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:13,418 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:13,418 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 7D82DFADB9454D12B60E57CCFD3BA219 on vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,419 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:13,419 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {7D82DFADB9454D12B60E57CCFD3BA219=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:13,421 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] parallelism set to 1
2016-05-18 00:39:13,421 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:13,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1C857C118D64F478D7F3881E3121865, vertexId=vertex_1463531953246_0001_1_00, initRequestedTime=1463531953408, initedTime=1463531953421, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:13,422 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] with distanceFromRoot: 0
2016-05-18 00:39:13,422 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:13,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531953246_0001_1_00, startRequestedTime=1463531953409, startedTime=1463531953422
2016-05-18 00:39:13,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531953246_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:13,424 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:13,424 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531953246_0001_1_00 for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] numStartedSources: 1 numSources: 1
2016-05-18 00:39:13,424 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] with distanceFromRoot: 1
2016-05-18 00:39:13,424 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D1C857C118D64F478D7F3881E3121865
2016-05-18 00:39:13,425 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531953246_0001_1_01, startRequestedTime=1463531953424, startedTime=1463531953425
2016-05-18 00:39:13,426 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:13,428 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D30F424DD9A8406F980F5352577249C2 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:13,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_STARTED]: vertexName=D1C857C118D64F478D7F3881E3121865, taskId=task_1463531953246_0001_1_00_000000, scheduledTime=1463531953428, launchTime=1463531953428
2016-05-18 00:39:13,429 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,429 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D1C857C118D64F478D7F3881E3121865 in vertex: D30F424DD9A8406F980F5352577249C2 numBipartiteSourceTasks: 1
2016-05-18 00:39:13,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531953246_0001, with dag id: dag_1463531953246_0001_1
2016-05-18 00:39:13,430 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:13,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531953246_0001_00_000001, launchTime=1463531953435
2016-05-18 00:39:13,437 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953246_0001_00_000001
2016-05-18 00:39:13,437 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531953246_0001_1_00_000000_0] started. Is using containerId: [container_1463531953246_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:13,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1C857C118D64F478D7F3881E3121865, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0, startTime=1463531953437, containerId=container_1463531953246_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:13,438 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953246_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13 Starting to run new task attempt: attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,440 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:13,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: D1C857C118D64F478D7F3881E3121865, VertexParallelism: 1, TaskAttemptID:attempt_1463531953246_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7D82DFADB9454D12B60E57CCFD3BA219, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D30F424DD9A8406F980F5352577249C2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:13,440 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:13,441 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,460 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:13,464 INFO input.MRInput (MRInput.java:initialize(440)) - 7D82DFADB9454D12B60E57CCFD3BA219 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:13,464 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 7D82DFADB9454D12B60E57CCFD3BA219
2016-05-18 00:39:13,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1C857C118D64F478D7F3881E3121865, ordinal: 0
2016-05-18 00:39:13,465 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1C857C118D64F478D7F3881E3121865, mem on start (mb), free: 277, total: 331, max: 672
2016-05-18 00:39:13,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:13,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:13,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:13,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:13,465 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[7D82DFADB9454D12B60E57CCFD3BA219:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D30F424DD9A8406F980F5352577249C2:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:13,465 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:13,465 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:13,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:13,466 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[7D82DFADB9454D12B60E57CCFD3BA219:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D30F424DD9A8406F980F5352577249C2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:13,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:13,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 7D82DFADB9454D12B60E57CCFD3BA219 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:13,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:13,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:13,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 7D82DFADB9454D12B60E57CCFD3BA219
2016-05-18 00:39:13,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:13,469 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,470 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531953246_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:13,471 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531953246_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:13,472 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:13,472 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 7D82DFADB9454D12B60E57CCFD3BA219 initialized RecordReader from event
2016-05-18 00:39:13,473 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:13,474 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 7D82DFADB9454D12B60E57CCFD3BA219
2016-05-18 00:39:13,474 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 3C4CA025B822493EB86E4E5CE6AEEABF
2016-05-18 00:39:13,474 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 3C4CA025B822493EB86E4E5CE6AEEABF
2016-05-18 00:39:13,477 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D30F424DD9A8406F980F5352577249C2 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@3c220740, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@5e0a1997, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:13,477 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D30F424DD9A8406F980F5352577249C2: , UsingHashComparator=false
2016-05-18 00:39:13,490 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:13,490 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:13,491 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D30F424DD9A8406F980F5352577249C2: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:13,491 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D30F424DD9A8406F980F5352577249C2: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:13,492 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:13,492 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1C857C118D64F478D7F3881E3121865, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:13,498 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,498 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1C857C118D64F478D7F3881E3121865, mem on close (mb), free: 176, total: 331, max: 672
2016-05-18 00:39:13,498 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D30F424DD9A8406F980F5352577249C2: Starting flush of map output
2016-05-18 00:39:13,499 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D30F424DD9A8406F980F5352577249C2: Span0.length = 37, perItem = 24
2016-05-18 00:39:13,500 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D30F424DD9A8406F980F5352577249C2: done sorting span=0, length=37, time=1
2016-05-18 00:39:13,500 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D30F424DD9A8406F980F5352577249C2: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:13,502 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D30F424DD9A8406F980F5352577249C2: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-local-dir/output/attempt_1463531953246_0001_1_00_000000_0_10125_0/file.out
2016-05-18 00:39:13,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:13 Completed running task attempt: attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,506 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:13,506 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531953246_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:13,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531953246_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531953492, Process_Duration=6, Process_End_Time=1463531953498, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:13,508 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531953246_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:13,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1C857C118D64F478D7F3881E3121865, taskAttemptId=attempt_1463531953246_0001_1_00_000000_0, creationTime=1463531953429, allocationTime=1463531953435, startTime=1463531953437, finishTime=1463531953508, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:13,508 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:13,508 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:13,508 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,509 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953246_0001_00_000001
2016-05-18 00:39:13,508 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531953246_0001_1_00_000000_0
2016-05-18 00:39:13,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_FINISHED]: vertexName=D1C857C118D64F478D7F3881E3121865, taskId=task_1463531953246_0001_1_00_000000, startTime=1463531953437, finishTime=1463531953509, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463531953246_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531953492, Process_Duration=6, Process_End_Time=1463531953498, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:13,510 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:13,510 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_FINISHED]: vertexName=D1C857C118D64F478D7F3881E3121865, vertexId=vertex_1463531953246_0001_1_00, initRequestedTime=1463531953408, initedTime=1463531953421, startRequestedTime=1463531953409, startedTime=1463531953422, finishTime=1463531953510, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531953492, Process_Duration=6, Process_End_Time=1463531953498, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531953437, firstTasksToStart=[ task_1463531953246_0001_1_00_000000 ], lastTaskFinishTime=1463531953508, lastTasksToFinish=[ task_1463531953246_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531953246_0001_1_00_000000 ], longestDurationTasks=[ task_1463531953246_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:13,511 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:13,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531953246_0001_00_000001, stoppedTime=1463531953511, exitStatus=0
2016-05-18 00:39:13,511 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] attempt: attempt_1463531953246_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:13,512 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:13,512 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:13,512 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531953246_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:13,512 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D30F424DD9A8406F980F5352577249C2
2016-05-18 00:39:13,512 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D1C857C118D64F478D7F3881E3121865, destinationVertexName=D30F424DD9A8406F980F5352577249C2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:13,512 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_STARTED]: vertexName=D30F424DD9A8406F980F5352577249C2, taskId=task_1463531953246_0001_1_01_000000, scheduledTime=1463531953513, launchTime=1463531953513
2016-05-18 00:39:13,513 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531953246_0001_00_000002, launchTime=1463531953514
2016-05-18 00:39:13,515 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953246_0001_00_000002
2016-05-18 00:39:13,516 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531953246_0001_1_01_000000_0] started. Is using containerId: [container_1463531953246_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:13,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D30F424DD9A8406F980F5352577249C2, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0, startTime=1463531953516, containerId=container_1463531953246_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:13,516 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953246_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13 Starting to run new task attempt: attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,517 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,518 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:13,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: D30F424DD9A8406F980F5352577249C2, VertexParallelism: 1, TaskAttemptID:attempt_1463531953246_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D1C857C118D64F478D7F3881E3121865, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3B37CEDA695244A1A886832A0F40EBA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:13,519 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:13,519 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531953246_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:13,519 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953246_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:13,519 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531953246_0001_00_000001, Exiting
2016-05-18 00:39:13,519 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531953246_0001_00_000001
2016-05-18 00:39:13,519 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531953246_0001_00_000001
2016-05-18 00:39:13,520 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,521 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531953246_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:13,521 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531953246_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,521 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531953246_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:13,522 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531953246_0001_00_000001 completed successfully
2016-05-18 00:39:13,524 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:13,527 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:13,533 INFO output.MROutput (MROutput.java:initialize(421)) - 3B37CEDA695244A1A886832A0F40EBA1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:13,534 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D30F424DD9A8406F980F5352577249C2, ordinal: 1
2016-05-18 00:39:13,535 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D30F424DD9A8406F980F5352577249C2, mem on start (mb), free: 171, total: 331, max: 672
2016-05-18 00:39:13,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:13,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:13,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:13,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:13,536 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[D1C857C118D64F478D7F3881E3121865:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [3B37CEDA695244A1A886832A0F40EBA1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:13,536 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:13,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:13,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:13,537 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[D1C857C118D64F478D7F3881E3121865:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [3B37CEDA695244A1A886832A0F40EBA1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:13,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:13,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D1C857C118D64F478D7F3881E3121865 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:13,538 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D1C857C118D64F478D7F3881E3121865: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:13,539 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D1C857C118D64F478D7F3881E3121865: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:13,540 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D1C857C118D64F478D7F3881E3121865 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:13,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D1C857C118D64F478D7F3881E3121865
2016-05-18 00:39:13,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:13,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:13,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:13,543 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,546 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:13,546 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D1C857C118D64F478D7F3881E3121865: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:13,547 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 3C4CA025B822493EB86E4E5CE6AEEABF
2016-05-18 00:39:13,549 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"], id: 3B37CEDA695244A1A886832A0F40EBA1
2016-05-18 00:39:13,549 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"]
2016-05-18 00:39:13,549 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 3C4CA025B822493EB86E4E5CE6AEEABF, for 1 inputs
2016-05-18 00:39:13,548 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531953246_0001_1_00_000000_0_10125} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531953548, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:13,550 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D1C857C118D64F478D7F3881E3121865
2016-05-18 00:39:13,550 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:13,550 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D1C857C118D64F478D7F3881E3121865, wasInterrupted=false
2016-05-18 00:39:13,551 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:13,552 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:13,553 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001_wd/localmode-local-dir/output/attempt_1463531953246_0001_1_00_000000_0_10125/file.out, len=732, isLocal=true
2016-05-18 00:39:13,553 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:13,554 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:13,554 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:13,554 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:13,555 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D30F424DD9A8406F980F5352577249C2, all 1 inputs ready in: 00:00:00.006
2016-05-18 00:39:13,555 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D1C857C118D64F478D7F3881E3121865
2016-05-18 00:39:13,556 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D1C857C118D64F478D7F3881E3121865: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:13,559 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D1C857C118D64F478D7F3881E3121865: Shuffle Runner thread complete
2016-05-18 00:39:13,560 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - D1C857C118D64F478D7F3881E3121865: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:13,563 INFO output.MROutput (MROutput.java:close(526)) - 3B37CEDA695244A1A886832A0F40EBA1 closed
2016-05-18 00:39:13,563 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531953246_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:13,564 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,564 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D30F424DD9A8406F980F5352577249C2, mem on close (mb), free: 168, total: 331, max: 672
2016-05-18 00:39:13,564 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D1C857C118D64F478D7F3881E3121865
2016-05-18 00:39:13,564 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D1C857C118D64F478D7F3881E3121865: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:13,564 INFO output.MROutput (MROutput.java:close(526)) - 3B37CEDA695244A1A886832A0F40EBA1 closed
2016-05-18 00:39:13,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:13 Completed running task attempt: attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,565 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:13,566 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531953246_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:13,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531953246_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531953555, Process_Duration=9, Process_End_Time=1463531953564, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:13,567 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531953246_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:13,567 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:13,567 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:13,567 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D30F424DD9A8406F980F5352577249C2, taskAttemptId=attempt_1463531953246_0001_1_01_000000_0, creationTime=1463531953513, allocationTime=1463531953514, startTime=1463531953516, finishTime=1463531953568, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:13,568 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531953246_0001_1_01_000000_0
2016-05-18 00:39:13,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:TASK_FINISHED]: vertexName=D30F424DD9A8406F980F5352577249C2, taskId=task_1463531953246_0001_1_01_000000, startTime=1463531953516, finishTime=1463531953568, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463531953246_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531953555, Process_Duration=9, Process_End_Time=1463531953564, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:13,569 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:13,569 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2]
2016-05-18 00:39:13,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:VERTEX_FINISHED]: vertexName=D30F424DD9A8406F980F5352577249C2, vertexId=vertex_1463531953246_0001_1_01, initRequestedTime=1463531953409, initedTime=1463531953413, startRequestedTime=1463531953424, startedTime=1463531953425, finishTime=1463531953569, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531953555, Process_Duration=9, Process_End_Time=1463531953564, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531953516, firstTasksToStart=[ task_1463531953246_0001_1_01_000000 ], lastTaskFinishTime=1463531953568, lastTasksToFinish=[ task_1463531953246_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531953246_0001_1_01_000000 ], longestDurationTasks=[ task_1463531953246_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:13,570 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:13,570 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:13,570 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:13,570 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531953246_0001_1_00 [D1C857C118D64F478D7F3881E3121865]
2016-05-18 00:39:13,570 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531953246_0001_1
2016-05-18 00:39:13,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531953246_0001_1
2016-05-18 00:39:13,571 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953246_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:13,571 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 3B37CEDA695244A1A886832A0F40EBA1 for vertex: vertex_1463531953246_0001_1_01 [D30F424DD9A8406F980F5352577249C2], outputName: 3B37CEDA695244A1A886832A0F40EBA1
2016-05-18 00:39:13,572 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953246_0001_00_000002
2016-05-18 00:39:13,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531953246_0001_00_000002, stoppedTime=1463531953572, exitStatus=0
2016-05-18 00:39:13,573 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531953246_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:13,580 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:3B37CEDA695244A1A886832A0F40EBA1 of vertex/vertexGroup:D30F424DD9A8406F980F5352577249C2 isVertexGroupOutput:false
2016-05-18 00:39:13,580 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:13,582 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531953246_0001_1, queueSize=0
2016-05-18 00:39:13,582 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531953246_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:13,582 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953246_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:13,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953246_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531953246_0001_1, startTime=1463531953407, finishTime=1463531953580, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=440, AM_GC_TIME_MILLIS=14, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063907047, Process_Duration=15, Process_End_Time=2927063907062, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:13,583 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531953246_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:13,583 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531953246_0001_00_000002, Exiting
2016-05-18 00:39:13,583 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531953246_0001_00_000002
2016-05-18 00:39:13,583 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531953246_0001_00_000002
2016-05-18 00:39:13,584 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531953246_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:13,584 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531953246_0001_00_000002 completed successfully
2016-05-18 00:39:13,583 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953246_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:13 Completed Dag: dag_1463531953246_0001_1
2016-05-18 00:39:13,584 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531953246_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:13,584 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531953246_0001_1_post
2016-05-18 00:39:13,585 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:39:13,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:13,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true, with id=dag_1463531953246_0001_1
2016-05-18 00:39:13,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4] comprehensivesort/(1/1) ...ive/true_false_false_true, with id=dag_1463531953246_0001_1
2016-05-18 00:39:13,591 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1463531953246_0001
2016-05-18 00:39:13,593 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:13,593 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:13,593 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:13,593 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:13,593 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:13,593 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:13,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:13,594 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:13,594 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:13,595 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:13,595 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:13,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:13,595 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:13,596 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:13,599 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/BA3F7A464C184E699A7E40CDAC49BBD2/BAF005047C554F1FABD1AE0D37F2A9A4/.tez/application_1463531953246_0001
2016-05-18 00:39:13,599 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true/_temporary
2016-05-18 00:39:13,600 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.373
2016-05-18 00:39:13,670 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:13,671 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:13,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-18 00:39:13,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-18 00:39:13,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:13,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:13,719 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:13,733 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:13,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true
2016-05-18 00:39:13,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:13,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:13,787 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"]
2016-05-18 00:39:13,787 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:13,787 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:13,787 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:13,788 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_false_true
2016-05-18 00:39:13,788 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:13,801 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:13,805 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:13,805 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:13,805 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:13,816 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001 doesn't exist and is created
2016-05-18 00:39:13,830 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:13,830 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:13,831 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd
2016-05-18 00:39:13,841 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531953805_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:13,841 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:13,841 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:13,843 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:13,843 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:13,843 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:13,843 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:13,844 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:13,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:13,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:13,846 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:13,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:13,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:13,847 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:13,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:13,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:13,847 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:13,848 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:13,848 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:13,849 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:13,850 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:13,851 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:13,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:13,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-log-dir
2016-05-18 00:39:13,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-log-dir/history.txt.appattempt_1463531953805_0001_000000, maxErrors=10
2016-05-18 00:39:13,852 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:13,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531953805_0001_000000, appSubmitTime=1463531953840, launchTime=1463531953841
2016-05-18 00:39:13,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:13,862 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54396
2016-05-18 00:39:13,862 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531953805
2016-05-18 00:39:13,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531953805_0001_000000, startTime=1463531953864
2016-05-18 00:39:13,864 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:13,898 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:13,899 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:13,930 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:13,931 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:13,931 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1463531953805_0001, dagName=[457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-05-18 00:39:13,947 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-05-18 00:39:13,948 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:13,948 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531953805_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-log-dir/dag_1463531953805_0001_1.dot
2016-05-18 00:39:13,949 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-log-dir/dag_1463531953805_0001_1-tez-dag.pb.txt
2016-05-18 00:39:13,955 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531953805_0001_1
2016-05-18 00:39:13,956 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:13,956 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-05-18 00:39:13 Running Dag: dag_1463531953805_0001_1
2016-05-18 00:39:13,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531953805_0001_1, submitTime=1463531953947
2016-05-18 00:39:13,968 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001/tez-conf.pb" } size: 36903 timestamp: 1463531953000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:13,968 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,968 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:13,969 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001/tez-conf.pb" } size: 36903 timestamp: 1463531953000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:13,969 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:13,969 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:13,971 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:13,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531953805_0001_1, initTime=1463531953971
2016-05-18 00:39:13,971 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953805_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:13,972 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:13,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:DAG_STARTED]: dagID=dag_1463531953805_0001_1, startTime=1463531953972
2016-05-18 00:39:13,972 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953805_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:13,972 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F5833FF4D976414B802117C4C0139BA2 : {63DE86D4ACBE474A9B74B840C0A28537={InputName=63DE86D4ACBE474A9B74B840C0A28537}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:13,972 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 63DE86D4ACBE474A9B74B840C0A28537, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:13,972 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,972 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,972 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,973 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,973 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:13,973 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:13,975 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:13,976 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:13,976 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:13,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531953805_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:13,976 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15], numAdditionalOutputs=1
2016-05-18 00:39:13,977 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=36B6086B809E4A7D9CE6BC2F2B88A8E1, vertex=vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:13,978 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 63DE86D4ACBE474A9B74B840C0A28537 on vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,978 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1463531953805_0001, dagName=[457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-05-18 00:39:13,979 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A3CB99F598BE436EA6CDC43ABB804F15:36B6086B809E4A7D9CE6BC2F2B88A8E1 using oldmapred API
2016-05-18 00:39:13,979 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=36B6086B809E4A7D9CE6BC2F2B88A8E1, vertexName=A3CB99F598BE436EA6CDC43ABB804F15, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:13,980 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 63DE86D4ACBE474A9B74B840C0A28537 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:13,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, vertexId=vertex_1463531953805_0001_1_01, initRequestedTime=1463531953973, initedTime=1463531953976, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:13,981 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:13,982 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:13,982 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:13,982 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:13,983 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:13,983 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 63DE86D4ACBE474A9B74B840C0A28537 on vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,984 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:13,984 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {63DE86D4ACBE474A9B74B840C0A28537=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:13,985 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] parallelism set to 1
2016-05-18 00:39:13,985 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:13,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F5833FF4D976414B802117C4C0139BA2, vertexId=vertex_1463531953805_0001_1_00, initRequestedTime=1463531953972, initedTime=1463531953986, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:13,986 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] with distanceFromRoot: 0
2016-05-18 00:39:13,986 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:13,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531953805_0001_1_00, startRequestedTime=1463531953973, startedTime=1463531953986
2016-05-18 00:39:13,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531953805_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:13,987 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:13,987 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531953805_0001_1_00 for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] numStartedSources: 1 numSources: 1
2016-05-18 00:39:13,987 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] with distanceFromRoot: 1
2016-05-18 00:39:13,987 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F5833FF4D976414B802117C4C0139BA2
2016-05-18 00:39:13,988 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531953805_0001_1_01, startRequestedTime=1463531953987, startedTime=1463531953987
2016-05-18 00:39:13,988 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:13,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_STARTED]: vertexName=F5833FF4D976414B802117C4C0139BA2, taskId=task_1463531953805_0001_1_00_000000, scheduledTime=1463531953988, launchTime=1463531953988
2016-05-18 00:39:13,989 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:13,990 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A3CB99F598BE436EA6CDC43ABB804F15 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:13,993 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: F5833FF4D976414B802117C4C0139BA2 in vertex: A3CB99F598BE436EA6CDC43ABB804F15 numBipartiteSourceTasks: 1
2016-05-18 00:39:13,993 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:14,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531953805_0001_00_000001, launchTime=1463531954008
2016-05-18 00:39:14,009 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953805_0001_00_000001
2016-05-18 00:39:14,010 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531953805_0001_1_00_000000_0] started. Is using containerId: [container_1463531953805_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:14,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F5833FF4D976414B802117C4C0139BA2, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0, startTime=1463531954009, containerId=container_1463531953805_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:14,010 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953805_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14 Starting to run new task attempt: attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,011 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,011 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531953805_0001, with dag id: dag_1463531953805_0001_1
2016-05-18 00:39:14,011 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,012 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:14,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: F5833FF4D976414B802117C4C0139BA2, VertexParallelism: 1, TaskAttemptID:attempt_1463531953805_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=63DE86D4ACBE474A9B74B840C0A28537, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A3CB99F598BE436EA6CDC43ABB804F15, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:14,013 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:14,013 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,014 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531953805_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:14,014 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531953805_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:14,017 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:14,018 INFO input.MRInput (MRInput.java:initialize(440)) - 63DE86D4ACBE474A9B74B840C0A28537 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:14,019 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 63DE86D4ACBE474A9B74B840C0A28537
2016-05-18 00:39:14,025 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F5833FF4D976414B802117C4C0139BA2, ordinal: 0
2016-05-18 00:39:14,025 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5833FF4D976414B802117C4C0139BA2, mem on start (mb), free: 277, total: 335, max: 672
2016-05-18 00:39:14,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:14,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:14,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:14,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:14,026 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[63DE86D4ACBE474A9B74B840C0A28537:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A3CB99F598BE436EA6CDC43ABB804F15:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:14,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:14,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:14,026 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:14,026 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[63DE86D4ACBE474A9B74B840C0A28537:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A3CB99F598BE436EA6CDC43ABB804F15:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:14,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:14,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 63DE86D4ACBE474A9B74B840C0A28537 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:14,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:14,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:14,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 63DE86D4ACBE474A9B74B840C0A28537
2016-05-18 00:39:14,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:14,029 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,032 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,033 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 63DE86D4ACBE474A9B74B840C0A28537
2016-05-18 00:39:14,033 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: D18FF7411F5A4AC98F438DB1670042B5
2016-05-18 00:39:14,034 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D18FF7411F5A4AC98F438DB1670042B5
2016-05-18 00:39:14,037 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:14,037 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 63DE86D4ACBE474A9B74B840C0A28537 initialized RecordReader from event
2016-05-18 00:39:14,038 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A3CB99F598BE436EA6CDC43ABB804F15 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@4d01f6c2, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@610e6238, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:14,039 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A3CB99F598BE436EA6CDC43ABB804F15: , UsingHashComparator=false
2016-05-18 00:39:14,053 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:14,053 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:14,053 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A3CB99F598BE436EA6CDC43ABB804F15: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:14,054 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A3CB99F598BE436EA6CDC43ABB804F15: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:14,054 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,054 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F5833FF4D976414B802117C4C0139BA2, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:14,059 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,059 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5833FF4D976414B802117C4C0139BA2, mem on close (mb), free: 175, total: 335, max: 672
2016-05-18 00:39:14,059 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A3CB99F598BE436EA6CDC43ABB804F15: Starting flush of map output
2016-05-18 00:39:14,060 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A3CB99F598BE436EA6CDC43ABB804F15: Span0.length = 37, perItem = 24
2016-05-18 00:39:14,062 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A3CB99F598BE436EA6CDC43ABB804F15: done sorting span=0, length=37, time=2
2016-05-18 00:39:14,063 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A3CB99F598BE436EA6CDC43ABB804F15: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:14,064 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A3CB99F598BE436EA6CDC43ABB804F15: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-local-dir/output/attempt_1463531953805_0001_1_00_000000_0_10130_0/file.out
2016-05-18 00:39:14,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:14 Completed running task attempt: attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,068 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:14,068 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531953805_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:14,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531953805_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531954055, Process_Duration=4, Process_End_Time=1463531954059, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:14,069 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531953805_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F5833FF4D976414B802117C4C0139BA2, taskAttemptId=attempt_1463531953805_0001_1_00_000000_0, creationTime=1463531953989, allocationTime=1463531953994, startTime=1463531954009, finishTime=1463531954069, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:14,069 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,070 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531953805_0001_1_00_000000_0
2016-05-18 00:39:14,070 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:14,070 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_FINISHED]: vertexName=F5833FF4D976414B802117C4C0139BA2, taskId=task_1463531953805_0001_1_00_000000, startTime=1463531954009, finishTime=1463531954070, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463531953805_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531954055, Process_Duration=4, Process_End_Time=1463531954059, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:14,071 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:14,071 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:14,071 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953805_0001_00_000001
2016-05-18 00:39:14,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_FINISHED]: vertexName=F5833FF4D976414B802117C4C0139BA2, vertexId=vertex_1463531953805_0001_1_00, initRequestedTime=1463531953972, initedTime=1463531953986, startRequestedTime=1463531953973, startedTime=1463531953986, finishTime=1463531954071, timeTaken=85, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531954055, Process_Duration=4, Process_End_Time=1463531954059, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531954009, firstTasksToStart=[ task_1463531953805_0001_1_00_000000 ], lastTaskFinishTime=1463531954069, lastTasksToFinish=[ task_1463531953805_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531953805_0001_1_00_000000 ], longestDurationTasks=[ task_1463531953805_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:14,072 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:14,072 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] attempt: attempt_1463531953805_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:14,072 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A3CB99F598BE436EA6CDC43ABB804F15
2016-05-18 00:39:14,072 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=F5833FF4D976414B802117C4C0139BA2, destinationVertexName=A3CB99F598BE436EA6CDC43ABB804F15. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:14,072 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:14,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531953805_0001_00_000001, stoppedTime=1463531954072, exitStatus=0
2016-05-18 00:39:14,073 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:14,073 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_STARTED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, taskId=task_1463531953805_0001_1_01_000000, scheduledTime=1463531954073, launchTime=1463531954073
2016-05-18 00:39:14,073 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531953805_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:14,073 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:14,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531953805_0001_00_000002, launchTime=1463531954075
2016-05-18 00:39:14,076 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953805_0001_00_000002
2016-05-18 00:39:14,076 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531953805_0001_1_01_000000_0] started. Is using containerId: [container_1463531953805_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:14,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0, startTime=1463531954076, containerId=container_1463531953805_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:14,077 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953805_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14 Starting to run new task attempt: attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,077 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,077 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,078 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:14,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: A3CB99F598BE436EA6CDC43ABB804F15, VertexParallelism: 1, TaskAttemptID:attempt_1463531953805_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F5833FF4D976414B802117C4C0139BA2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=36B6086B809E4A7D9CE6BC2F2B88A8E1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:14,078 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:14,078 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,079 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531953805_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:14,080 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531953805_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:14,081 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:14,082 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531953805_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:14,082 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953805_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:14,082 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531953805_0001_00_000001, Exiting
2016-05-18 00:39:14,082 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531953805_0001_00_000001
2016-05-18 00:39:14,082 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531953805_0001_00_000001
2016-05-18 00:39:14,085 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3CB99F598BE436EA6CDC43ABB804F15, ordinal: 1
2016-05-18 00:39:14,085 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3CB99F598BE436EA6CDC43ABB804F15, mem on start (mb), free: 172, total: 335, max: 672
2016-05-18 00:39:14,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:14,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:14,086 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531953805_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:14,087 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531953805_0001_00_000001 completed successfully
2016-05-18 00:39:14,088 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:14,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:14,090 INFO output.MROutput (MROutput.java:initialize(421)) - 36B6086B809E4A7D9CE6BC2F2B88A8E1: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:14,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:14,091 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[36B6086B809E4A7D9CE6BC2F2B88A8E1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [F5833FF4D976414B802117C4C0139BA2:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:14,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:14,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:14,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:14,091 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[36B6086B809E4A7D9CE6BC2F2B88A8E1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [F5833FF4D976414B802117C4C0139BA2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:14,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:14,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F5833FF4D976414B802117C4C0139BA2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:14,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:14,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:14,092 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - F5833FF4D976414B802117C4C0139BA2: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:14,092 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - F5833FF4D976414B802117C4C0139BA2: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:14,093 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: F5833FF4D976414B802117C4C0139BA2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:14,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F5833FF4D976414B802117C4C0139BA2
2016-05-18 00:39:14,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:14,097 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,099 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F5833FF4D976414B802117C4C0139BA2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:14,101 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531953805_0001_1_00_000000_0_10130} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531954101, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:14,101 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:14,101 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : F5833FF4D976414B802117C4C0139BA2
2016-05-18 00:39:14,101 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:14,101 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: D18FF7411F5A4AC98F438DB1670042B5
2016-05-18 00:39:14,101 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: F5833FF4D976414B802117C4C0139BA2, wasInterrupted=false
2016-05-18 00:39:14,101 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"], id: 36B6086B809E4A7D9CE6BC2F2B88A8E1
2016-05-18 00:39:14,102 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:14,102 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"]
2016-05-18 00:39:14,102 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:14,102 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] D18FF7411F5A4AC98F438DB1670042B5, for 1 inputs
2016-05-18 00:39:14,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001_wd/localmode-local-dir/output/attempt_1463531953805_0001_1_00_000000_0_10130/file.out, len=732, isLocal=true
2016-05-18 00:39:14,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:14,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:14,104 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:14,104 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:14,104 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : F5833FF4D976414B802117C4C0139BA2
2016-05-18 00:39:14,104 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - F5833FF4D976414B802117C4C0139BA2: Shuffle Runner thread complete
2016-05-18 00:39:14,105 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3CB99F598BE436EA6CDC43ABB804F15, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:14,106 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - F5833FF4D976414B802117C4C0139BA2: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:14,110 INFO output.MROutput (MROutput.java:close(526)) - 36B6086B809E4A7D9CE6BC2F2B88A8E1 closed
2016-05-18 00:39:14,110 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531953805_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:14,111 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,111 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3CB99F598BE436EA6CDC43ABB804F15, mem on close (mb), free: 168, total: 335, max: 672
2016-05-18 00:39:14,111 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: F5833FF4D976414B802117C4C0139BA2
2016-05-18 00:39:14,112 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F5833FF4D976414B802117C4C0139BA2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:14,112 INFO output.MROutput (MROutput.java:close(526)) - 36B6086B809E4A7D9CE6BC2F2B88A8E1 closed
2016-05-18 00:39:14,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:14 Completed running task attempt: attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,113 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:14,113 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531953805_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:14,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531953805_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531954105, Process_Duration=6, Process_End_Time=1463531954111, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:14,114 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531953805_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, taskAttemptId=attempt_1463531953805_0001_1_01_000000_0, creationTime=1463531954073, allocationTime=1463531954074, startTime=1463531954076, finishTime=1463531954114, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:14,115 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531953805_0001_1_01_000000_0
2016-05-18 00:39:14,114 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,115 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:14,115 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:TASK_FINISHED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, taskId=task_1463531953805_0001_1_01_000000, startTime=1463531954076, finishTime=1463531954115, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463531953805_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531954105, Process_Duration=6, Process_End_Time=1463531954111, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:14,115 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:14,115 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15]
2016-05-18 00:39:14,116 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531953805_0001_00_000002
2016-05-18 00:39:14,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:VERTEX_FINISHED]: vertexName=A3CB99F598BE436EA6CDC43ABB804F15, vertexId=vertex_1463531953805_0001_1_01, initRequestedTime=1463531953973, initedTime=1463531953976, startRequestedTime=1463531953987, startedTime=1463531953987, finishTime=1463531954116, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531954105, Process_Duration=6, Process_End_Time=1463531954111, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531954076, firstTasksToStart=[ task_1463531953805_0001_1_01_000000 ], lastTaskFinishTime=1463531954114, lastTasksToFinish=[ task_1463531953805_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531953805_0001_1_01_000000 ], longestDurationTasks=[ task_1463531953805_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:14,117 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:14,117 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:14,117 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,117 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531953805_0001_1_00 [F5833FF4D976414B802117C4C0139BA2]
2016-05-18 00:39:14,117 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531953805_0001_1
2016-05-18 00:39:14,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531953805_0001_1
2016-05-18 00:39:14,117 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953805_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:14,118 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 36B6086B809E4A7D9CE6BC2F2B88A8E1 for vertex: vertex_1463531953805_0001_1_01 [A3CB99F598BE436EA6CDC43ABB804F15], outputName: 36B6086B809E4A7D9CE6BC2F2B88A8E1
2016-05-18 00:39:14,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531953805_0001_00_000002, stoppedTime=1463531954118, exitStatus=0
2016-05-18 00:39:14,118 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531953805_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:14,118 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531953805_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:14,118 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531953805_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:14,118 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531953805_0001_00_000002, Exiting
2016-05-18 00:39:14,118 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531953805_0001_00_000002
2016-05-18 00:39:14,118 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531953805_0001_00_000002
2016-05-18 00:39:14,119 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531953805_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:14,119 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531953805_0001_00_000002 completed successfully
2016-05-18 00:39:14,124 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:36B6086B809E4A7D9CE6BC2F2B88A8E1 of vertex/vertexGroup:A3CB99F598BE436EA6CDC43ABB804F15 isVertexGroupOutput:false
2016-05-18 00:39:14,125 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,126 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531953805_0001_1, queueSize=0
2016-05-18 00:39:14,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531953805_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531953805_0001_1, startTime=1463531953972, finishTime=1463531954125, timeTaken=153, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=320, AM_GC_TIME_MILLIS=13, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063908160, Process_Duration=10, Process_End_Time=2927063908170, Read_Duration=5, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:14,127 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531953805_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:14,127 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531953805_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:14 Completed Dag: dag_1463531953805_0001_1
2016-05-18 00:39:14,127 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531953805_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:14,127 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531953805_0001_1_post
2016-05-18 00:39:14,127 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,127 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:14,127 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:14,128 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true, with id=dag_1463531953805_0001_1
2016-05-18 00:39:14,128 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927] comprehensivesort/(1/1) ...sive/true_true_false_true, with id=dag_1463531953805_0001_1
2016-05-18 00:39:14,144 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1463531953805_0001
2016-05-18 00:39:14,145 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:14,145 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:14,145 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:14,145 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:14,145 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:14,146 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:14,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:14,147 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:14,147 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:14,147 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:14,147 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:14,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:14,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:14,148 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:14,152 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/457816C2718148DEAD1FCFA5E27B0514/C8A57EC3DFB2468D87AB8DFA3AA3F927/.tez/application_1463531953805_0001
2016-05-18 00:39:14,153 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true/_temporary
2016-05-18 00:39:14,153 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.367
2016-05-18 00:39:14,222 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:14,223 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:14,256 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-18 00:39:14,262 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:39:14,263 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:14,263 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:14,263 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:14,278 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:14,284 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true
2016-05-18 00:39:14,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:14,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,334 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"]
2016-05-18 00:39:14,334 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:14,335 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:14,335 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:14,337 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_false_true
2016-05-18 00:39:14,337 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:14,350 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:14,355 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:14,355 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:14,355 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:14,365 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001 doesn't exist and is created
2016-05-18 00:39:14,379 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:14,379 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:14,380 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd
2016-05-18 00:39:14,389 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531954355_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:14,389 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:14,389 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:14,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:14,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:14,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:14,391 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:14,392 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:14,393 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:14,393 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:14,394 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:14,394 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:14,394 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:14,394 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:14,394 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:14,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:14,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:14,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:14,395 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:14,396 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:14,397 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:14,398 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:14,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:14,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-log-dir
2016-05-18 00:39:14,399 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-log-dir/history.txt.appattempt_1463531954355_0001_000000, maxErrors=10
2016-05-18 00:39:14,399 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:14,399 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531954355_0001_000000, appSubmitTime=1463531954388, launchTime=1463531954388
2016-05-18 00:39:14,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:14,405 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57591
2016-05-18 00:39:14,405 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531954355
2016-05-18 00:39:14,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531954355_0001_000000, startTime=1463531954407
2016-05-18 00:39:14,407 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:14,480 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:14,480 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:14,480 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1463531954355_0001, dagName=[C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-05-18 00:39:14,496 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-05-18 00:39:14,496 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:14,497 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531954355_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-log-dir/dag_1463531954355_0001_1.dot
2016-05-18 00:39:14,498 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-log-dir/dag_1463531954355_0001_1-tez-dag.pb.txt
2016-05-18 00:39:14,504 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531954355_0001_1
2016-05-18 00:39:14,504 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,505 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-05-18 00:39:14 Running Dag: dag_1463531954355_0001_1
2016-05-18 00:39:14,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531954355_0001_1, submitTime=1463531954496
2016-05-18 00:39:14,515 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001/tez-conf.pb" } size: 36903 timestamp: 1463531954000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:14,515 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,515 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:14,516 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001/tez-conf.pb" } size: 36903 timestamp: 1463531954000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:14,517 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,517 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:14,518 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:14,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531954355_0001_1, initTime=1463531954518
2016-05-18 00:39:14,519 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954355_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:14,519 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:14,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:DAG_STARTED]: dagID=dag_1463531954355_0001_1, startTime=1463531954519
2016-05-18 00:39:14,519 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954355_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: C5DB99D325F24E0EB9577E5892898094 : {1CD3A01E197844F5A3C491F2D053BE9E={InputName=1CD3A01E197844F5A3C491F2D053BE9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 1CD3A01E197844F5A3C491F2D053BE9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,520 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,520 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1463531954355_0001, dagName=[C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-05-18 00:39:14,521 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:14,521 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,522 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 1CD3A01E197844F5A3C491F2D053BE9E on vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,543 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 1CD3A01E197844F5A3C491F2D053BE9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:14,544 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:14,545 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:14,545 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:14,545 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:14,545 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:14,546 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 1CD3A01E197844F5A3C491F2D053BE9E on vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,546 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,548 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531954355_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:14,549 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4], numAdditionalOutputs=1
2016-05-18 00:39:14,550 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=EAAA3631AA0F4BB1B70B725E939BEFDF, vertex=vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:14,551 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for EE7B3E294CE64FD8B6C7B5BDF493DCC4:EAAA3631AA0F4BB1B70B725E939BEFDF using oldmapred API
2016-05-18 00:39:14,552 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=EAAA3631AA0F4BB1B70B725E939BEFDF, vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:14,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, vertexId=vertex_1463531954355_0001_1_01, initRequestedTime=1463531954521, initedTime=1463531954549, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:14,553 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:14,554 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:14,554 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {1CD3A01E197844F5A3C491F2D053BE9E=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:14,554 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] parallelism set to 1
2016-05-18 00:39:14,554 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:14,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C5DB99D325F24E0EB9577E5892898094, vertexId=vertex_1463531954355_0001_1_00, initRequestedTime=1463531954519, initedTime=1463531954555, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:14,555 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] with distanceFromRoot: 0
2016-05-18 00:39:14,556 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:14,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531954355_0001_1_00, startRequestedTime=1463531954521, startedTime=1463531954556
2016-05-18 00:39:14,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531954355_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:14,557 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:14,557 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:14,557 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in C5DB99D325F24E0EB9577E5892898094
2016-05-18 00:39:14,557 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:14,557 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531954355_0001, with dag id: dag_1463531954355_0001_1
2016-05-18 00:39:14,557 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,558 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531954355_0001_1_00 for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] numStartedSources: 1 numSources: 1
2016-05-18 00:39:14,558 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] with distanceFromRoot: 1
2016-05-18 00:39:14,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531954355_0001_1_01, startRequestedTime=1463531954557, startedTime=1463531954558
2016-05-18 00:39:14,559 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:14,559 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: EE7B3E294CE64FD8B6C7B5BDF493DCC4 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:14,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_STARTED]: vertexName=C5DB99D325F24E0EB9577E5892898094, taskId=task_1463531954355_0001_1_00_000000, scheduledTime=1463531954559, launchTime=1463531954559
2016-05-18 00:39:14,560 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,561 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: C5DB99D325F24E0EB9577E5892898094 in vertex: EE7B3E294CE64FD8B6C7B5BDF493DCC4 numBipartiteSourceTasks: 1
2016-05-18 00:39:14,562 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:14,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531954355_0001_00_000001, launchTime=1463531954564
2016-05-18 00:39:14,564 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954355_0001_00_000001
2016-05-18 00:39:14,565 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531954355_0001_1_00_000000_0] started. Is using containerId: [container_1463531954355_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:14,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C5DB99D325F24E0EB9577E5892898094, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0, startTime=1463531954565, containerId=container_1463531954355_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:14,565 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954355_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14 Starting to run new task attempt: attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,566 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,566 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,566 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:14,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: C5DB99D325F24E0EB9577E5892898094, VertexParallelism: 1, TaskAttemptID:attempt_1463531954355_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1CD3A01E197844F5A3C491F2D053BE9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:14,567 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:14,567 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,569 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531954355_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:14,569 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531954355_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:14,570 INFO input.MRInput (MRInput.java:initialize(440)) - 1CD3A01E197844F5A3C491F2D053BE9E using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:14,570 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 1CD3A01E197844F5A3C491F2D053BE9E
2016-05-18 00:39:14,570 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:14,577 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C5DB99D325F24E0EB9577E5892898094, ordinal: 0
2016-05-18 00:39:14,577 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5DB99D325F24E0EB9577E5892898094, mem on start (mb), free: 264, total: 328, max: 672
2016-05-18 00:39:14,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:14,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:14,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:14,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:14,577 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[1CD3A01E197844F5A3C491F2D053BE9E:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [EE7B3E294CE64FD8B6C7B5BDF493DCC4:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:14,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:14,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:14,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:14,578 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[1CD3A01E197844F5A3C491F2D053BE9E:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [EE7B3E294CE64FD8B6C7B5BDF493DCC4:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:14,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:14,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 1CD3A01E197844F5A3C491F2D053BE9E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:14,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 1CD3A01E197844F5A3C491F2D053BE9E
2016-05-18 00:39:14,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:14,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:14,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:14,579 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,580 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:14,580 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 1CD3A01E197844F5A3C491F2D053BE9E initialized RecordReader from event
2016-05-18 00:39:14,583 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,583 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 1CD3A01E197844F5A3C491F2D053BE9E
2016-05-18 00:39:14,583 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 463F03654E1542B78DF2F48C4BD5FE9C
2016-05-18 00:39:14,584 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 463F03654E1542B78DF2F48C4BD5FE9C
2016-05-18 00:39:14,586 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@76c3ff8, comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@53dbb696, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:14,586 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for EE7B3E294CE64FD8B6C7B5BDF493DCC4: , UsingHashComparator=false
2016-05-18 00:39:14,598 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:14,598 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:14,599 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for EE7B3E294CE64FD8B6C7B5BDF493DCC4: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:14,599 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:14,599 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,599 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C5DB99D325F24E0EB9577E5892898094, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:14,606 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,606 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5DB99D325F24E0EB9577E5892898094, mem on close (mb), free: 162, total: 328, max: 672
2016-05-18 00:39:14,606 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: Starting flush of map output
2016-05-18 00:39:14,606 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: Span0.length = 37, perItem = 24
2016-05-18 00:39:14,607 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: done sorting span=0, length=37, time=1
2016-05-18 00:39:14,607 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:14,608 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - EE7B3E294CE64FD8B6C7B5BDF493DCC4: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-local-dir/output/attempt_1463531954355_0001_1_00_000000_0_10137_0/file.out
2016-05-18 00:39:14,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:14 Completed running task attempt: attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,612 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:14,612 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531954355_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:14,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531954355_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531954599, Process_Duration=7, Process_End_Time=1463531954606, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:14,613 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531954355_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C5DB99D325F24E0EB9577E5892898094, taskAttemptId=attempt_1463531954355_0001_1_00_000000_0, creationTime=1463531954560, allocationTime=1463531954563, startTime=1463531954565, finishTime=1463531954613, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:14,613 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,614 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531954355_0001_1_00_000000_0
2016-05-18 00:39:14,614 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:14,614 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_FINISHED]: vertexName=C5DB99D325F24E0EB9577E5892898094, taskId=task_1463531954355_0001_1_00_000000, startTime=1463531954565, finishTime=1463531954614, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463531954355_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531954599, Process_Duration=7, Process_End_Time=1463531954606, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:14,614 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954355_0001_00_000001
2016-05-18 00:39:14,614 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:14,614 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_FINISHED]: vertexName=C5DB99D325F24E0EB9577E5892898094, vertexId=vertex_1463531954355_0001_1_00, initRequestedTime=1463531954519, initedTime=1463531954555, startRequestedTime=1463531954521, startedTime=1463531954556, finishTime=1463531954615, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531954599, Process_Duration=7, Process_End_Time=1463531954606, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531954565, firstTasksToStart=[ task_1463531954355_0001_1_00_000000 ], lastTaskFinishTime=1463531954613, lastTasksToFinish=[ task_1463531954355_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531954355_0001_1_00_000000 ], longestDurationTasks=[ task_1463531954355_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:14,617 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:14,617 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] attempt: attempt_1463531954355_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:14,617 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EE7B3E294CE64FD8B6C7B5BDF493DCC4
2016-05-18 00:39:14,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531954355_0001_00_000001, stoppedTime=1463531954617, exitStatus=0
2016-05-18 00:39:14,617 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=C5DB99D325F24E0EB9577E5892898094, destinationVertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:14,617 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,617 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:14,618 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531954355_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:14,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_STARTED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, taskId=task_1463531954355_0001_1_01_000000, scheduledTime=1463531954619, launchTime=1463531954619
2016-05-18 00:39:14,619 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531954355_0001_00_000002, launchTime=1463531954620
2016-05-18 00:39:14,621 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954355_0001_00_000002
2016-05-18 00:39:14,621 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531954355_0001_1_01_000000_0] started. Is using containerId: [container_1463531954355_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:14,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0, startTime=1463531954621, containerId=container_1463531954355_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:14,622 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954355_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14 Starting to run new task attempt: attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,622 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,623 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:14,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: EE7B3E294CE64FD8B6C7B5BDF493DCC4, VertexParallelism: 1, TaskAttemptID:attempt_1463531954355_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C5DB99D325F24E0EB9577E5892898094, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EAAA3631AA0F4BB1B70B725E939BEFDF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:14,623 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:14,624 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,625 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531954355_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:14,625 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954355_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:14,625 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531954355_0001_00_000001, Exiting
2016-05-18 00:39:14,625 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531954355_0001_00_000001
2016-05-18 00:39:14,625 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531954355_0001_00_000001
2016-05-18 00:39:14,627 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:14,629 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:14,630 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531954355_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,631 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531954355_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:14,631 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531954355_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:14,631 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531954355_0001_00_000001 completed successfully
2016-05-18 00:39:14,633 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE7B3E294CE64FD8B6C7B5BDF493DCC4, ordinal: 1
2016-05-18 00:39:14,633 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE7B3E294CE64FD8B6C7B5BDF493DCC4, mem on start (mb), free: 157, total: 328, max: 672
2016-05-18 00:39:14,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:14,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:14,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:14,636 INFO output.MROutput (MROutput.java:initialize(421)) - EAAA3631AA0F4BB1B70B725E939BEFDF: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:14,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:14,636 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[EAAA3631AA0F4BB1B70B725E939BEFDF:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [C5DB99D325F24E0EB9577E5892898094:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:14,636 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:14,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:14,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:14,637 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[EAAA3631AA0F4BB1B70B725E939BEFDF:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [C5DB99D325F24E0EB9577E5892898094:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:14,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:14,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: C5DB99D325F24E0EB9577E5892898094 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:14,638 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - C5DB99D325F24E0EB9577E5892898094: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:14,638 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - C5DB99D325F24E0EB9577E5892898094: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:14,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: C5DB99D325F24E0EB9577E5892898094 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:14,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:14,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:14,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: C5DB99D325F24E0EB9577E5892898094
2016-05-18 00:39:14,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:14,641 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,643 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:14,644 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 463F03654E1542B78DF2F48C4BD5FE9C
2016-05-18 00:39:14,644 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"], id: EAAA3631AA0F4BB1B70B725E939BEFDF
2016-05-18 00:39:14,644 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"]
2016-05-18 00:39:14,644 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 463F03654E1542B78DF2F48C4BD5FE9C, for 1 inputs
2016-05-18 00:39:14,644 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C5DB99D325F24E0EB9577E5892898094: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:14,646 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531954355_0001_1_00_000000_0_10137} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531954646, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:14,646 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : C5DB99D325F24E0EB9577E5892898094
2016-05-18 00:39:14,647 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:14,647 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: C5DB99D325F24E0EB9577E5892898094, wasInterrupted=false
2016-05-18 00:39:14,648 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:14,648 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:14,649 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001_wd/localmode-local-dir/output/attempt_1463531954355_0001_1_00_000000_0_10137/file.out, len=732, isLocal=true
2016-05-18 00:39:14,649 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:14,650 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:14,650 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:14,650 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:14,650 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : C5DB99D325F24E0EB9577E5892898094
2016-05-18 00:39:14,650 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE7B3E294CE64FD8B6C7B5BDF493DCC4, all 1 inputs ready in: 00:00:00.006
2016-05-18 00:39:14,650 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - C5DB99D325F24E0EB9577E5892898094: Shuffle Runner thread complete
2016-05-18 00:39:14,651 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - C5DB99D325F24E0EB9577E5892898094: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.GroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:14,655 INFO output.MROutput (MROutput.java:close(526)) - EAAA3631AA0F4BB1B70B725E939BEFDF closed
2016-05-18 00:39:14,656 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531954355_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:14,656 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,656 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE7B3E294CE64FD8B6C7B5BDF493DCC4, mem on close (mb), free: 154, total: 328, max: 672
2016-05-18 00:39:14,657 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: C5DB99D325F24E0EB9577E5892898094
2016-05-18 00:39:14,657 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - C5DB99D325F24E0EB9577E5892898094: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:14,657 INFO output.MROutput (MROutput.java:close(526)) - EAAA3631AA0F4BB1B70B725E939BEFDF closed
2016-05-18 00:39:14,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:14 Completed running task attempt: attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,658 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:14,658 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531954355_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:14,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531954355_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531954651, Process_Duration=5, Process_End_Time=1463531954656, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:14,659 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531954355_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,659 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:14,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, taskAttemptId=attempt_1463531954355_0001_1_01_000000_0, creationTime=1463531954619, allocationTime=1463531954620, startTime=1463531954621, finishTime=1463531954659, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:14,659 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:14,659 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,660 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531954355_0001_1_01_000000_0
2016-05-18 00:39:14,660 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954355_0001_00_000002
2016-05-18 00:39:14,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:TASK_FINISHED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, taskId=task_1463531954355_0001_1_01_000000, startTime=1463531954621, finishTime=1463531954660, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463531954355_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531954651, Process_Duration=5, Process_End_Time=1463531954656, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:14,660 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:14,660 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4]
2016-05-18 00:39:14,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:VERTEX_FINISHED]: vertexName=EE7B3E294CE64FD8B6C7B5BDF493DCC4, vertexId=vertex_1463531954355_0001_1_01, initRequestedTime=1463531954521, initedTime=1463531954549, startRequestedTime=1463531954557, startedTime=1463531954558, finishTime=1463531954661, timeTaken=103, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531954651, Process_Duration=5, Process_End_Time=1463531954656, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531954621, firstTasksToStart=[ task_1463531954355_0001_1_01_000000 ], lastTaskFinishTime=1463531954659, lastTasksToFinish=[ task_1463531954355_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531954355_0001_1_01_000000 ], longestDurationTasks=[ task_1463531954355_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:14,661 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:14,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531954355_0001_00_000002, stoppedTime=1463531954662, exitStatus=0
2016-05-18 00:39:14,662 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:14,662 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,662 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531954355_0001_1_00 [C5DB99D325F24E0EB9577E5892898094]
2016-05-18 00:39:14,662 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531954355_0001_1
2016-05-18 00:39:14,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531954355_0001_1
2016-05-18 00:39:14,662 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: EAAA3631AA0F4BB1B70B725E939BEFDF for vertex: vertex_1463531954355_0001_1_01 [EE7B3E294CE64FD8B6C7B5BDF493DCC4], outputName: EAAA3631AA0F4BB1B70B725E939BEFDF
2016-05-18 00:39:14,664 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954355_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:14,664 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531954355_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:14,668 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:EAAA3631AA0F4BB1B70B725E939BEFDF of vertex/vertexGroup:EE7B3E294CE64FD8B6C7B5BDF493DCC4 isVertexGroupOutput:false
2016-05-18 00:39:14,668 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:14,670 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531954355_0001_1, queueSize=0
2016-05-18 00:39:14,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954355_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531954355_0001_1, startTime=1463531954519, finishTime=1463531954668, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=260, AM_GC_TIME_MILLIS=14, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063909250, Process_Duration=12, Process_End_Time=2927063909262, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:14,670 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531954355_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:14,671 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954355_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:14 Completed Dag: dag_1463531954355_0001_1
2016-05-18 00:39:14,671 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531954355_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:14,671 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531954355_0001_1_post
2016-05-18 00:39:14,671 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:14,671 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:14,671 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:14,672 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true, with id=dag_1463531954355_0001_1
2016-05-18 00:39:14,672 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531954355_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:14,672 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97] comprehensivesort/(1/1) ...ive/false_true_false_true, with id=dag_1463531954355_0001_1
2016-05-18 00:39:14,672 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954355_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:14,672 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531954355_0001_00_000002, Exiting
2016-05-18 00:39:14,672 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531954355_0001_00_000002
2016-05-18 00:39:14,672 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531954355_0001_00_000002
2016-05-18 00:39:14,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531954355_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:14,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531954355_0001_00_000002 completed successfully
2016-05-18 00:39:14,679 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1463531954355_0001
2016-05-18 00:39:14,681 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:14,681 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:14,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:14,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:14,681 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:14,681 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:14,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:14,682 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:14,682 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:14,682 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:14,683 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:14,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:14,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:14,685 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:14,688 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/C3DBC0696FE64B91BE14301AF812DC72/D871CD8F4AF441A48DFC8D0A0D1E8B97/.tez/application_1463531954355_0001
2016-05-18 00:39:14,689 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true/_temporary
2016-05-18 00:39:14,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.356
2016-05-18 00:39:14,755 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:14,756 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:14,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:39:14,794 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-18 00:39:14,794 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:14,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:14,795 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:14,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:14,825 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true
2016-05-18 00:39:14,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:14,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:14,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"]
2016-05-18 00:39:14,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:14,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:14,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:14,888 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...ive/false_false_true_true
2016-05-18 00:39:14,888 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:14,901 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:14,905 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:14,906 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:14,906 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:14,914 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001 doesn't exist and is created
2016-05-18 00:39:14,928 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:14,929 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:14,929 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd
2016-05-18 00:39:14,938 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531954905_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:14,939 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:14,939 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:14,941 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:14,941 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:14,941 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:14,941 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:14,941 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:14,943 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:14,943 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:14,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:14,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:14,944 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:14,945 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:14,945 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:14,945 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:14,945 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:14,945 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:14,947 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:14,947 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:14,948 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:14,949 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:14,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:14,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-log-dir
2016-05-18 00:39:14,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-log-dir/history.txt.appattempt_1463531954905_0001_000000, maxErrors=10
2016-05-18 00:39:14,950 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:14,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531954905_0001_000000, appSubmitTime=1463531954938, launchTime=1463531954938
2016-05-18 00:39:14,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:14,959 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44107
2016-05-18 00:39:14,959 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531954905
2016-05-18 00:39:14,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531954905_0001_000000, startTime=1463531954961
2016-05-18 00:39:14,961 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:15,029 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:15,029 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:15,029 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1463531954905_0001, dagName=[9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-05-18 00:39:15,045 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-05-18 00:39:15,045 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:15,046 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531954905_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-log-dir/dag_1463531954905_0001_1.dot
2016-05-18 00:39:15,046 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-log-dir/dag_1463531954905_0001_1-tez-dag.pb.txt
2016-05-18 00:39:15,068 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531954905_0001_1
2016-05-18 00:39:15,068 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,069 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-05-18 00:39:15 Running Dag: dag_1463531954905_0001_1
2016-05-18 00:39:15,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531954905_0001_1, submitTime=1463531955045
2016-05-18 00:39:15,078 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001/tez-conf.pb" } size: 36903 timestamp: 1463531954000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:15,079 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,079 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:15,081 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001/tez-conf.pb" } size: 36903 timestamp: 1463531954000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:15,081 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,081 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:15,082 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:15,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531954905_0001_1, initTime=1463531955083
2016-05-18 00:39:15,084 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954905_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:15,084 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:15,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:DAG_STARTED]: dagID=dag_1463531954905_0001_1, startTime=1463531955084
2016-05-18 00:39:15,084 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954905_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:15,084 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1463531954905_0001, dagName=[9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-05-18 00:39:15,084 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: A4E5A6C9C8A74DD08DA0891CB3053DBD : {57A65E2B6AB14400BBD1ADA555A2DB2F={InputName=57A65E2B6AB14400BBD1ADA555A2DB2F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:15,086 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 57A65E2B6AB14400BBD1ADA555A2DB2F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:15,086 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,086 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,086 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,086 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,087 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:15,087 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,087 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 57A65E2B6AB14400BBD1ADA555A2DB2F on vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,089 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 57A65E2B6AB14400BBD1ADA555A2DB2F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:15,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:15,091 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:15,092 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:15,092 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:15,093 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 57A65E2B6AB14400BBD1ADA555A2DB2F on vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,091 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:15,093 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,094 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531954905_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:15,095 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448], numAdditionalOutputs=1
2016-05-18 00:39:15,095 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=0E7EE03385804445879FAE1689F5B7AC, vertex=vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:15,096 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for BF4D6605D8C944248100ADBDE66F1448:0E7EE03385804445879FAE1689F5B7AC using oldmapred API
2016-05-18 00:39:15,097 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=0E7EE03385804445879FAE1689F5B7AC, vertexName=BF4D6605D8C944248100ADBDE66F1448, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:15,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, vertexId=vertex_1463531954905_0001_1_01, initRequestedTime=1463531955087, initedTime=1463531955095, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:15,098 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:15,098 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:15,098 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {57A65E2B6AB14400BBD1ADA555A2DB2F=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:15,100 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] parallelism set to 1
2016-05-18 00:39:15,101 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:15,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, vertexId=vertex_1463531954905_0001_1_00, initRequestedTime=1463531955084, initedTime=1463531955101, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:15,101 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] with distanceFromRoot: 0
2016-05-18 00:39:15,102 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:15,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531954905_0001_1_00, startRequestedTime=1463531955087, startedTime=1463531955102
2016-05-18 00:39:15,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531954905_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:15,103 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:15,103 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531954905_0001_1_00 for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] numStartedSources: 1 numSources: 1
2016-05-18 00:39:15,103 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] with distanceFromRoot: 1
2016-05-18 00:39:15,103 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in A4E5A6C9C8A74DD08DA0891CB3053DBD
2016-05-18 00:39:15,104 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531954905_0001_1_01, startRequestedTime=1463531955103, startedTime=1463531955104
2016-05-18 00:39:15,107 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:15,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_STARTED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, taskId=task_1463531954905_0001_1_00_000000, scheduledTime=1463531955107, launchTime=1463531955107
2016-05-18 00:39:15,108 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,108 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: BF4D6605D8C944248100ADBDE66F1448 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:15,109 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:15,110 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: A4E5A6C9C8A74DD08DA0891CB3053DBD in vertex: BF4D6605D8C944248100ADBDE66F1448 numBipartiteSourceTasks: 1
2016-05-18 00:39:15,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531954905_0001_00_000001, launchTime=1463531955112
2016-05-18 00:39:15,113 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954905_0001_00_000001
2016-05-18 00:39:15,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531954905_0001_1_00_000000_0] started. Is using containerId: [container_1463531954905_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:15,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0, startTime=1463531955113, containerId=container_1463531954905_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:15,114 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954905_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531954905_0001, with dag id: dag_1463531954905_0001_1
2016-05-18 00:39:15 Starting to run new task attempt: attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,115 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:15,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: A4E5A6C9C8A74DD08DA0891CB3053DBD, VertexParallelism: 1, TaskAttemptID:attempt_1463531954905_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=57A65E2B6AB14400BBD1ADA555A2DB2F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF4D6605D8C944248100ADBDE66F1448, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:15,117 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:15,118 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,118 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531954905_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:15,119 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531954905_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:15,121 INFO input.MRInput (MRInput.java:initialize(440)) - 57A65E2B6AB14400BBD1ADA555A2DB2F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:15,122 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 57A65E2B6AB14400BBD1ADA555A2DB2F
2016-05-18 00:39:15,122 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:15,126 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A4E5A6C9C8A74DD08DA0891CB3053DBD, ordinal: 0
2016-05-18 00:39:15,126 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4E5A6C9C8A74DD08DA0891CB3053DBD, mem on start (mb), free: 261, total: 334, max: 672
2016-05-18 00:39:15,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:15,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:15,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:15,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:15,127 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[57A65E2B6AB14400BBD1ADA555A2DB2F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [BF4D6605D8C944248100ADBDE66F1448:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:15,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:15,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:15,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:15,129 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[57A65E2B6AB14400BBD1ADA555A2DB2F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [BF4D6605D8C944248100ADBDE66F1448:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:15,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:15,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 57A65E2B6AB14400BBD1ADA555A2DB2F
2016-05-18 00:39:15,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 57A65E2B6AB14400BBD1ADA555A2DB2F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:15,129 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:15,130 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:15,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:15,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:15,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:15,131 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,134 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:15,134 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 57A65E2B6AB14400BBD1ADA555A2DB2F initialized RecordReader from event
2016-05-18 00:39:15,135 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,136 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 57A65E2B6AB14400BBD1ADA555A2DB2F
2016-05-18 00:39:15,136 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 6731A6E6477548A39F6BF964F2F324C8
2016-05-18 00:39:15,136 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 6731A6E6477548A39F6BF964F2F324C8
2016-05-18 00:39:15,138 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - BF4D6605D8C944248100ADBDE66F1448 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@2393366a, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@15d97b65, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:15,138 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for BF4D6605D8C944248100ADBDE66F1448: , UsingHashComparator=false
2016-05-18 00:39:15,151 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:15,151 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:15,151 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for BF4D6605D8C944248100ADBDE66F1448: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:15,151 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - BF4D6605D8C944248100ADBDE66F1448: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:15,152 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,152 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A4E5A6C9C8A74DD08DA0891CB3053DBD, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:15,157 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,157 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A4E5A6C9C8A74DD08DA0891CB3053DBD, mem on close (mb), free: 159, total: 334, max: 672
2016-05-18 00:39:15,157 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - BF4D6605D8C944248100ADBDE66F1448: Starting flush of map output
2016-05-18 00:39:15,157 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - BF4D6605D8C944248100ADBDE66F1448: Span0.length = 37, perItem = 24
2016-05-18 00:39:15,159 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - BF4D6605D8C944248100ADBDE66F1448: done sorting span=0, length=37, time=2
2016-05-18 00:39:15,159 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - BF4D6605D8C944248100ADBDE66F1448: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:15,160 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - BF4D6605D8C944248100ADBDE66F1448: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-local-dir/output/attempt_1463531954905_0001_1_00_000000_0_10143_0/file.out
2016-05-18 00:39:15,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:15 Completed running task attempt: attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,164 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:15,164 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531954905_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:15,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531954905_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531955152, Process_Duration=5, Process_End_Time=1463531955157, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:15,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, taskAttemptId=attempt_1463531954905_0001_1_00_000000_0, creationTime=1463531955108, allocationTime=1463531955111, startTime=1463531955113, finishTime=1463531955165, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:15,165 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531954905_0001_1_00_000000_0
2016-05-18 00:39:15,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_FINISHED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, taskId=task_1463531954905_0001_1_00_000000, startTime=1463531955113, finishTime=1463531955166, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463531954905_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531955152, Process_Duration=5, Process_End_Time=1463531955157, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:15,166 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:15,167 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_FINISHED]: vertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, vertexId=vertex_1463531954905_0001_1_00, initRequestedTime=1463531955084, initedTime=1463531955101, startRequestedTime=1463531955087, startedTime=1463531955102, finishTime=1463531955167, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531955152, Process_Duration=5, Process_End_Time=1463531955157, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531955113, firstTasksToStart=[ task_1463531954905_0001_1_00_000000 ], lastTaskFinishTime=1463531955165, lastTasksToFinish=[ task_1463531954905_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531954905_0001_1_00_000000 ], longestDurationTasks=[ task_1463531954905_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:15,168 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:15,168 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] attempt: attempt_1463531954905_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:15,168 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531954905_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531954905_0001_00_000001, stoppedTime=1463531955168, exitStatus=0
2016-05-18 00:39:15,168 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BF4D6605D8C944248100ADBDE66F1448
2016-05-18 00:39:15,169 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:15,170 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,169 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,171 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531954905_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:15,169 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, destinationVertexName=BF4D6605D8C944248100ADBDE66F1448. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:15,172 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,172 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:15,172 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_STARTED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, taskId=task_1463531954905_0001_1_01_000000, scheduledTime=1463531955172, launchTime=1463531955172
2016-05-18 00:39:15,173 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954905_0001_00_000001
2016-05-18 00:39:15,173 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,173 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531954905_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:15,174 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954905_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:15,174 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531954905_0001_00_000001, Exiting
2016-05-18 00:39:15,174 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531954905_0001_00_000001
2016-05-18 00:39:15,174 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531954905_0001_00_000001
2016-05-18 00:39:15,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531954905_0001_00_000002, launchTime=1463531955175
2016-05-18 00:39:15,175 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531954905_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:15,176 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531954905_0001_00_000001 completed successfully
2016-05-18 00:39:15,176 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954905_0001_00_000002
2016-05-18 00:39:15,176 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954905_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,176 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531954905_0001_1_01_000000_0] started. Is using containerId: [container_1463531954905_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:15,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0, startTime=1463531955176, containerId=container_1463531954905_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:15 Starting to run new task attempt: attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,177 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,177 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,177 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:15,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: BF4D6605D8C944248100ADBDE66F1448, VertexParallelism: 1, TaskAttemptID:attempt_1463531954905_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A4E5A6C9C8A74DD08DA0891CB3053DBD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0E7EE03385804445879FAE1689F5B7AC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:15,178 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:15,178 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,179 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531954905_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,181 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531954905_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:15,182 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:15,184 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:15,187 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF4D6605D8C944248100ADBDE66F1448, ordinal: 1
2016-05-18 00:39:15,188 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF4D6605D8C944248100ADBDE66F1448, mem on start (mb), free: 155, total: 334, max: 672
2016-05-18 00:39:15,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:15,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:15,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:15,191 INFO output.MROutput (MROutput.java:initialize(421)) - 0E7EE03385804445879FAE1689F5B7AC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:15,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:15,191 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[A4E5A6C9C8A74DD08DA0891CB3053DBD:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [0E7EE03385804445879FAE1689F5B7AC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:15,191 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:15,191 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:15,191 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:15,191 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[A4E5A6C9C8A74DD08DA0891CB3053DBD:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [0E7EE03385804445879FAE1689F5B7AC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:15,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:15,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: A4E5A6C9C8A74DD08DA0891CB3053DBD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:15,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:15,192 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:15,192 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:15,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:15,193 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: A4E5A6C9C8A74DD08DA0891CB3053DBD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:15,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: A4E5A6C9C8A74DD08DA0891CB3053DBD
2016-05-18 00:39:15,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:15,194 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,195 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:15,198 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:15,199 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 6731A6E6477548A39F6BF964F2F324C8
2016-05-18 00:39:15,199 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"], id: 0E7EE03385804445879FAE1689F5B7AC
2016-05-18 00:39:15,199 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531954905_0001_1_00_000000_0_10143} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531955199, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:15,199 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : A4E5A6C9C8A74DD08DA0891CB3053DBD
2016-05-18 00:39:15,199 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"]
2016-05-18 00:39:15,200 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:15,200 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: A4E5A6C9C8A74DD08DA0891CB3053DBD, wasInterrupted=false
2016-05-18 00:39:15,200 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 6731A6E6477548A39F6BF964F2F324C8, for 1 inputs
2016-05-18 00:39:15,202 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:15,202 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:15,203 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001_wd/localmode-local-dir/output/attempt_1463531954905_0001_1_00_000000_0_10143/file.out, len=732, isLocal=true
2016-05-18 00:39:15,203 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:15,204 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:15,204 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:15,204 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:15,204 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : A4E5A6C9C8A74DD08DA0891CB3053DBD
2016-05-18 00:39:15,205 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF4D6605D8C944248100ADBDE66F1448, all 1 inputs ready in: 00:00:00.003
2016-05-18 00:39:15,205 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: Shuffle Runner thread complete
2016-05-18 00:39:15,206 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:15,210 INFO output.MROutput (MROutput.java:close(526)) - 0E7EE03385804445879FAE1689F5B7AC closed
2016-05-18 00:39:15,211 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531954905_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:15,211 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,211 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF4D6605D8C944248100ADBDE66F1448, mem on close (mb), free: 152, total: 334, max: 672
2016-05-18 00:39:15,211 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: A4E5A6C9C8A74DD08DA0891CB3053DBD
2016-05-18 00:39:15,212 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - A4E5A6C9C8A74DD08DA0891CB3053DBD: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:15,212 INFO output.MROutput (MROutput.java:close(526)) - 0E7EE03385804445879FAE1689F5B7AC closed
2016-05-18 00:39:15,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:15 Completed running task attempt: attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,213 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:15,213 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531954905_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:15,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531954905_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531955205, Process_Duration=6, Process_End_Time=1463531955211, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:15,214 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531954905_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, taskAttemptId=attempt_1463531954905_0001_1_01_000000_0, creationTime=1463531955173, allocationTime=1463531955174, startTime=1463531955176, finishTime=1463531955214, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:15,214 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,214 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:15,214 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531954905_0001_1_01_000000_0
2016-05-18 00:39:15,214 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:TASK_FINISHED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, taskId=task_1463531954905_0001_1_01_000000, startTime=1463531955176, finishTime=1463531955214, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463531954905_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531955205, Process_Duration=6, Process_End_Time=1463531955211, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:15,215 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:15,215 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448]
2016-05-18 00:39:15,215 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531954905_0001_00_000002
2016-05-18 00:39:15,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:VERTEX_FINISHED]: vertexName=BF4D6605D8C944248100ADBDE66F1448, vertexId=vertex_1463531954905_0001_1_01, initRequestedTime=1463531955087, initedTime=1463531955095, startRequestedTime=1463531955103, startedTime=1463531955104, finishTime=1463531955215, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531955205, Process_Duration=6, Process_End_Time=1463531955211, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531955176, firstTasksToStart=[ task_1463531954905_0001_1_01_000000 ], lastTaskFinishTime=1463531955214, lastTasksToFinish=[ task_1463531954905_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531954905_0001_1_01_000000 ], longestDurationTasks=[ task_1463531954905_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:15,217 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:15,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531954905_0001_00_000002, stoppedTime=1463531955217, exitStatus=0
2016-05-18 00:39:15,217 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:15,217 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,217 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531954905_0001_1_00 [A4E5A6C9C8A74DD08DA0891CB3053DBD]
2016-05-18 00:39:15,217 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531954905_0001_1
2016-05-18 00:39:15,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531954905_0001_1
2016-05-18 00:39:15,218 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954905_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:15,218 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 0E7EE03385804445879FAE1689F5B7AC for vertex: vertex_1463531954905_0001_1_01 [BF4D6605D8C944248100ADBDE66F1448], outputName: 0E7EE03385804445879FAE1689F5B7AC
2016-05-18 00:39:15,218 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531954905_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:15,224 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:0E7EE03385804445879FAE1689F5B7AC of vertex/vertexGroup:BF4D6605D8C944248100ADBDE66F1448 isVertexGroupOutput:false
2016-05-18 00:39:15,224 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,226 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531954905_0001_1, queueSize=0
2016-05-18 00:39:15,226 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531954905_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:15,226 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531954905_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:15,226 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531954905_0001_00_000002, Exiting
2016-05-18 00:39:15,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531954905_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531954905_0001_1, startTime=1463531955084, finishTime=1463531955224, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=230, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063910357, Process_Duration=11, Process_End_Time=2927063910368, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:15,226 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531954905_0001_00_000002
2016-05-18 00:39:15,226 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531954905_0001_00_000002
2016-05-18 00:39:15,227 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531954905_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:15,227 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531954905_0001_00_000002 completed successfully
2016-05-18 00:39:15,226 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531954905_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:15,227 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531954905_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:15 Completed Dag: dag_1463531954905_0001_1
2016-05-18 00:39:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531954905_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:15,228 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531954905_0001_1_post
2016-05-18 00:39:15,228 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:39:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true, with id=dag_1463531954905_0001_1
2016-05-18 00:39:15,228 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812] comprehensivesort/(1/1) ...ive/false_false_true_true, with id=dag_1463531954905_0001_1
2016-05-18 00:39:15,237 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1463531954905_0001
2016-05-18 00:39:15,239 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:15,239 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:15,239 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:15,239 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:15,239 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:15,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:15,240 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:15,240 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:15,240 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:15,241 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:15,241 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:15,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:15,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:15,242 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:15,246 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/9B830551D01C43EAB3B2E8D94A812838/E23513674C2E4A968ADFD25183428812/.tez/application_1463531954905_0001
2016-05-18 00:39:15,246 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true/_temporary
2016-05-18 00:39:15,246 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.361
2016-05-18 00:39:15,316 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:15,316 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:15,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-18 00:39:15,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-18 00:39:15,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:15,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:15,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:15,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:15,390 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true
2016-05-18 00:39:15,436 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:15,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"]
2016-05-18 00:39:15,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:15,437 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:15,438 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:15,438 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/true_false_true_true
2016-05-18 00:39:15,439 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:15,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:15,455 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:15,456 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:15,456 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:15,465 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001 doesn't exist and is created
2016-05-18 00:39:15,479 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:15,479 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:15,479 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd
2016-05-18 00:39:15,489 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531955456_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:15,489 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:15,489 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:15,491 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:15,491 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:15,491 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:15,491 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:15,492 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:15,493 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:15,494 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:15,494 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:15,495 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:15,495 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:15,496 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:15,496 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:15,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:15,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-log-dir
2016-05-18 00:39:15,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-log-dir/history.txt.appattempt_1463531955456_0001_000000, maxErrors=10
2016-05-18 00:39:15,497 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:15,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531955456_0001_000000, appSubmitTime=1463531955489, launchTime=1463531955489
2016-05-18 00:39:15,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:15,520 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51430
2016-05-18 00:39:15,520 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531955456
2016-05-18 00:39:15,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531955456_0001_000000, startTime=1463531955522
2016-05-18 00:39:15,522 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:15,579 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:15,579 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:15,579 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1463531955456_0001, dagName=[EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-05-18 00:39:15,597 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-05-18 00:39:15,597 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:15,598 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531955456_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-log-dir/dag_1463531955456_0001_1.dot
2016-05-18 00:39:15,599 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-log-dir/dag_1463531955456_0001_1-tez-dag.pb.txt
2016-05-18 00:39:15,607 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531955456_0001_1
2016-05-18 00:39:15,608 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,608 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-05-18 00:39:15 Running Dag: dag_1463531955456_0001_1
2016-05-18 00:39:15,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531955456_0001_1, submitTime=1463531955597
2016-05-18 00:39:15,619 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001/tez-conf.pb" } size: 36903 timestamp: 1463531955000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:15,619 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,619 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:15,620 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001/tez-conf.pb" } size: 36903 timestamp: 1463531955000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:15,620 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,620 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:15,621 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:15,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531955456_0001_1, initTime=1463531955621
2016-05-18 00:39:15,621 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531955456_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:15,622 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:15,622 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1463531955456_0001, dagName=[EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-05-18 00:39:15,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:DAG_STARTED]: dagID=dag_1463531955456_0001_1, startTime=1463531955622
2016-05-18 00:39:15,622 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531955456_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:15,622 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: F6CF50D6A632423BB4B433A626D32F31 : {2A84E2E5900444CCAE74F1008C474AB7={InputName=2A84E2E5900444CCAE74F1008C474AB7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:15,622 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 2A84E2E5900444CCAE74F1008C474AB7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:15,622 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,623 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,623 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,623 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,623 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:15,624 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 2A84E2E5900444CCAE74F1008C474AB7 on vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,624 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,626 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 2A84E2E5900444CCAE74F1008C474AB7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:15,627 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:15,627 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,627 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,627 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:15,628 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:15,629 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:15,629 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:15,629 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 2A84E2E5900444CCAE74F1008C474AB7 on vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531955456_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:15,636 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E], numAdditionalOutputs=1
2016-05-18 00:39:15,636 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=55A74C966B6444CC9706AA8B53AC0269, vertex=vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:15,638 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for D816C01773BA470A804CB37ECE1DDB9E:55A74C966B6444CC9706AA8B53AC0269 using oldmapred API
2016-05-18 00:39:15,638 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=55A74C966B6444CC9706AA8B53AC0269, vertexName=D816C01773BA470A804CB37ECE1DDB9E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:15,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, vertexId=vertex_1463531955456_0001_1_01, initRequestedTime=1463531955624, initedTime=1463531955636, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:15,640 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:15,640 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:15,641 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {2A84E2E5900444CCAE74F1008C474AB7=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:15,641 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] parallelism set to 1
2016-05-18 00:39:15,642 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:15,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, vertexId=vertex_1463531955456_0001_1_00, initRequestedTime=1463531955622, initedTime=1463531955642, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:15,642 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] with distanceFromRoot: 0
2016-05-18 00:39:15,642 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:15,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531955456_0001_1_00, startRequestedTime=1463531955624, startedTime=1463531955642
2016-05-18 00:39:15,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531955456_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:15,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531955456_0001, with dag id: dag_1463531955456_0001_1
2016-05-18 00:39:15,643 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:15,643 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531955456_0001_1_00 for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] numStartedSources: 1 numSources: 1
2016-05-18 00:39:15,643 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] with distanceFromRoot: 1
2016-05-18 00:39:15,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531955456_0001_1_01, startRequestedTime=1463531955643, startedTime=1463531955643
2016-05-18 00:39:15,644 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:15,645 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in F6CF50D6A632423BB4B433A626D32F31
2016-05-18 00:39:15,645 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_STARTED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, taskId=task_1463531955456_0001_1_00_000000, scheduledTime=1463531955645, launchTime=1463531955645
2016-05-18 00:39:15,646 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,655 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: D816C01773BA470A804CB37ECE1DDB9E with 0 source tasks and 1 pending tasks
2016-05-18 00:39:15,656 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: F6CF50D6A632423BB4B433A626D32F31 in vertex: D816C01773BA470A804CB37ECE1DDB9E numBipartiteSourceTasks: 1
2016-05-18 00:39:15,656 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:15,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531955456_0001_00_000001, launchTime=1463531955658
2016-05-18 00:39:15,659 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531955456_0001_00_000001
2016-05-18 00:39:15,660 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531955456_0001_1_00_000000_0] started. Is using containerId: [container_1463531955456_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:15,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0, startTime=1463531955660, containerId=container_1463531955456_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:15,661 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531955456_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15 Starting to run new task attempt: attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,661 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,661 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,662 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:15,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: F6CF50D6A632423BB4B433A626D32F31, VertexParallelism: 1, TaskAttemptID:attempt_1463531955456_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2A84E2E5900444CCAE74F1008C474AB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D816C01773BA470A804CB37ECE1DDB9E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:15,662 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:15,663 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,664 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531955456_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:15,664 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531955456_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:15,667 INFO input.MRInput (MRInput.java:initialize(440)) - 2A84E2E5900444CCAE74F1008C474AB7 using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:15,668 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 2A84E2E5900444CCAE74F1008C474AB7
2016-05-18 00:39:15,671 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:15,676 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F6CF50D6A632423BB4B433A626D32F31, ordinal: 0
2016-05-18 00:39:15,676 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6CF50D6A632423BB4B433A626D32F31, mem on start (mb), free: 252, total: 334, max: 672
2016-05-18 00:39:15,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:15,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:15,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:15,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:15,676 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[2A84E2E5900444CCAE74F1008C474AB7:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [D816C01773BA470A804CB37ECE1DDB9E:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:15,676 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:15,676 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:15,677 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:15,677 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[2A84E2E5900444CCAE74F1008C474AB7:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [D816C01773BA470A804CB37ECE1DDB9E:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 2A84E2E5900444CCAE74F1008C474AB7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 2A84E2E5900444CCAE74F1008C474AB7
2016-05-18 00:39:15,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:15,678 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,679 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:15,679 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 2A84E2E5900444CCAE74F1008C474AB7 initialized RecordReader from event
2016-05-18 00:39:15,681 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,681 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 2A84E2E5900444CCAE74F1008C474AB7
2016-05-18 00:39:15,681 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 409DB7E412484401B4EA26291DE5BA2B
2016-05-18 00:39:15,682 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 409DB7E412484401B4EA26291DE5BA2B
2016-05-18 00:39:15,683 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - D816C01773BA470A804CB37ECE1DDB9E using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@3b918ec9, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@51354cac, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:15,684 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for D816C01773BA470A804CB37ECE1DDB9E: , UsingHashComparator=false
2016-05-18 00:39:15,698 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:15,698 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:15,699 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for D816C01773BA470A804CB37ECE1DDB9E: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:15,699 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - D816C01773BA470A804CB37ECE1DDB9E: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:15,699 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,700 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F6CF50D6A632423BB4B433A626D32F31, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:15,704 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,705 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6CF50D6A632423BB4B433A626D32F31, mem on close (mb), free: 150, total: 334, max: 672
2016-05-18 00:39:15,705 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - D816C01773BA470A804CB37ECE1DDB9E: Starting flush of map output
2016-05-18 00:39:15,705 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - D816C01773BA470A804CB37ECE1DDB9E: Span0.length = 37, perItem = 24
2016-05-18 00:39:15,706 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - D816C01773BA470A804CB37ECE1DDB9E: done sorting span=0, length=37, time=1
2016-05-18 00:39:15,706 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - D816C01773BA470A804CB37ECE1DDB9E: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:15,707 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - D816C01773BA470A804CB37ECE1DDB9E: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-local-dir/output/attempt_1463531955456_0001_1_00_000000_0_10149_0/file.out
2016-05-18 00:39:15,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:15 Completed running task attempt: attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,711 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:15,711 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531955456_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:15,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531955456_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531955700, Process_Duration=4, Process_End_Time=1463531955704, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:15,712 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531955456_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,712 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, taskAttemptId=attempt_1463531955456_0001_1_00_000000_0, creationTime=1463531955646, allocationTime=1463531955657, startTime=1463531955660, finishTime=1463531955712, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:15,713 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531955456_0001_1_00_000000_0
2016-05-18 00:39:15,713 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:15,713 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_FINISHED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, taskId=task_1463531955456_0001_1_00_000000, startTime=1463531955660, finishTime=1463531955713, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463531955456_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531955700, Process_Duration=4, Process_End_Time=1463531955704, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:15,714 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:15,714 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,714 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531955456_0001_00_000001
2016-05-18 00:39:15,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_FINISHED]: vertexName=F6CF50D6A632423BB4B433A626D32F31, vertexId=vertex_1463531955456_0001_1_00, initRequestedTime=1463531955622, initedTime=1463531955642, startRequestedTime=1463531955624, startedTime=1463531955642, finishTime=1463531955714, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531955700, Process_Duration=4, Process_End_Time=1463531955704, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531955660, firstTasksToStart=[ task_1463531955456_0001_1_00_000000 ], lastTaskFinishTime=1463531955712, lastTasksToFinish=[ task_1463531955456_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531955456_0001_1_00_000000 ], longestDurationTasks=[ task_1463531955456_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:15,715 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:15,715 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] attempt: attempt_1463531955456_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:15,716 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D816C01773BA470A804CB37ECE1DDB9E
2016-05-18 00:39:15,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531955456_0001_00_000001, stoppedTime=1463531955716, exitStatus=0
2016-05-18 00:39:15,716 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=F6CF50D6A632423BB4B433A626D32F31, destinationVertexName=D816C01773BA470A804CB37ECE1DDB9E. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:15,716 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,716 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:15,716 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_STARTED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, taskId=task_1463531955456_0001_1_01_000000, scheduledTime=1463531955716, launchTime=1463531955716
2016-05-18 00:39:15,717 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531955456_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:15,717 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531955456_0001_00_000002, launchTime=1463531955718
2016-05-18 00:39:15,719 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531955456_0001_00_000002
2016-05-18 00:39:15,719 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531955456_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,719 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531955456_0001_1_01_000000_0] started. Is using containerId: [container_1463531955456_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:15,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0, startTime=1463531955719, containerId=container_1463531955456_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:15 Starting to run new task attempt: attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,720 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,720 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,720 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:15,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: D816C01773BA470A804CB37ECE1DDB9E, VertexParallelism: 1, TaskAttemptID:attempt_1463531955456_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6CF50D6A632423BB4B433A626D32F31, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=55A74C966B6444CC9706AA8B53AC0269, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:15,721 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:15,721 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,721 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531955456_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,723 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531955456_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:15,724 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:15,725 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531955456_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:15,726 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531955456_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:15,726 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531955456_0001_00_000001, Exiting
2016-05-18 00:39:15,726 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531955456_0001_00_000001
2016-05-18 00:39:15,726 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531955456_0001_00_000001
2016-05-18 00:39:15,728 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:15,729 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531955456_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:15,730 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531955456_0001_00_000001 completed successfully
2016-05-18 00:39:15,731 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D816C01773BA470A804CB37ECE1DDB9E, ordinal: 1
2016-05-18 00:39:15,731 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D816C01773BA470A804CB37ECE1DDB9E, mem on start (mb), free: 145, total: 334, max: 672
2016-05-18 00:39:15,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:15,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:15,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:15,733 INFO output.MROutput (MROutput.java:initialize(421)) - 55A74C966B6444CC9706AA8B53AC0269: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:15,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:15,734 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[55A74C966B6444CC9706AA8B53AC0269:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [F6CF50D6A632423BB4B433A626D32F31:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:15,734 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:15,734 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:15,734 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:15,734 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[55A74C966B6444CC9706AA8B53AC0269:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [F6CF50D6A632423BB4B433A626D32F31:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:15,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:15,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: F6CF50D6A632423BB4B433A626D32F31 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:15,735 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - F6CF50D6A632423BB4B433A626D32F31: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:15,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:15,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:15,735 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - F6CF50D6A632423BB4B433A626D32F31: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:15,736 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: F6CF50D6A632423BB4B433A626D32F31 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:15,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: F6CF50D6A632423BB4B433A626D32F31
2016-05-18 00:39:15,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:15,739 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,740 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:15,740 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:15,740 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F6CF50D6A632423BB4B433A626D32F31: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:15,743 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531955456_0001_1_00_000000_0_10149} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531955743, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:15,743 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : F6CF50D6A632423BB4B433A626D32F31
2016-05-18 00:39:15,744 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:15,744 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: F6CF50D6A632423BB4B433A626D32F31, wasInterrupted=false
2016-05-18 00:39:15,745 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:15,746 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:onSuccess(1328)) - F6CF50D6A632423BB4B433A626D32F31: Already shutdown. Ignoring fetch complete
2016-05-18 00:39:15,745 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:15,746 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:15,746 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 409DB7E412484401B4EA26291DE5BA2B
2016-05-18 00:39:15,746 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"], id: 55A74C966B6444CC9706AA8B53AC0269
2016-05-18 00:39:15,747 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"]
2016-05-18 00:39:15,747 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 409DB7E412484401B4EA26291DE5BA2B, for 1 inputs
2016-05-18 00:39:15,747 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001_wd/localmode-local-dir/output/attempt_1463531955456_0001_1_00_000000_0_10149/file.out, len=732, isLocal=true
2016-05-18 00:39:15,747 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:15,748 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:15,748 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:15,748 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:15,749 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : F6CF50D6A632423BB4B433A626D32F31
2016-05-18 00:39:15,749 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D816C01773BA470A804CB37ECE1DDB9E, all 1 inputs ready in: 00:00:00.002
2016-05-18 00:39:15,749 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - F6CF50D6A632423BB4B433A626D32F31: Shuffle Runner thread complete
2016-05-18 00:39:15,750 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - F6CF50D6A632423BB4B433A626D32F31: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:15,754 INFO output.MROutput (MROutput.java:close(526)) - 55A74C966B6444CC9706AA8B53AC0269 closed
2016-05-18 00:39:15,754 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531955456_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:15,755 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,755 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D816C01773BA470A804CB37ECE1DDB9E, mem on close (mb), free: 143, total: 334, max: 672
2016-05-18 00:39:15,755 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: F6CF50D6A632423BB4B433A626D32F31
2016-05-18 00:39:15,755 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - F6CF50D6A632423BB4B433A626D32F31: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:15,755 INFO output.MROutput (MROutput.java:close(526)) - 55A74C966B6444CC9706AA8B53AC0269 closed
2016-05-18 00:39:15,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:15 Completed running task attempt: attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,755 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:15,756 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531955456_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:15,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531955456_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531955749, Process_Duration=6, Process_End_Time=1463531955755, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:15,757 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531955456_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, taskAttemptId=attempt_1463531955456_0001_1_01_000000_0, creationTime=1463531955717, allocationTime=1463531955718, startTime=1463531955719, finishTime=1463531955757, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:15,757 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531955456_0001_1_01_000000_0
2016-05-18 00:39:15,757 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:15,757 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:15,757 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,758 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531955456_0001_00_000002
2016-05-18 00:39:15,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:TASK_FINISHED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, taskId=task_1463531955456_0001_1_01_000000, startTime=1463531955719, finishTime=1463531955757, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463531955456_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531955749, Process_Duration=6, Process_End_Time=1463531955755, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:15,760 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:15,760 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E]
2016-05-18 00:39:15,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:VERTEX_FINISHED]: vertexName=D816C01773BA470A804CB37ECE1DDB9E, vertexId=vertex_1463531955456_0001_1_01, initRequestedTime=1463531955624, initedTime=1463531955636, startRequestedTime=1463531955643, startedTime=1463531955643, finishTime=1463531955760, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531955749, Process_Duration=6, Process_End_Time=1463531955755, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531955719, firstTasksToStart=[ task_1463531955456_0001_1_01_000000 ], lastTaskFinishTime=1463531955757, lastTasksToFinish=[ task_1463531955456_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531955456_0001_1_01_000000 ], longestDurationTasks=[ task_1463531955456_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:15,761 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:15,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531955456_0001_00_000002, stoppedTime=1463531955761, exitStatus=0
2016-05-18 00:39:15,761 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:15,761 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,761 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531955456_0001_1_00 [F6CF50D6A632423BB4B433A626D32F31]
2016-05-18 00:39:15,761 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531955456_0001_1
2016-05-18 00:39:15,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531955456_0001_1
2016-05-18 00:39:15,762 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531955456_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:15,762 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 55A74C966B6444CC9706AA8B53AC0269 for vertex: vertex_1463531955456_0001_1_01 [D816C01773BA470A804CB37ECE1DDB9E], outputName: 55A74C966B6444CC9706AA8B53AC0269
2016-05-18 00:39:15,762 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531955456_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:15,767 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:55A74C966B6444CC9706AA8B53AC0269 of vertex/vertexGroup:D816C01773BA470A804CB37ECE1DDB9E isVertexGroupOutput:false
2016-05-18 00:39:15,767 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:15,768 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531955456_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:15,768 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531955456_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:15,769 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531955456_0001_00_000002, Exiting
2016-05-18 00:39:15,769 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531955456_0001_00_000002
2016-05-18 00:39:15,769 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531955456_0001_1, queueSize=0
2016-05-18 00:39:15,769 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531955456_0001_00_000002
2016-05-18 00:39:15,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531955456_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531955456_0001_1, startTime=1463531955622, finishTime=1463531955767, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=300, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063911449, Process_Duration=10, Process_End_Time=2927063911459, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:15,769 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531955456_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:15,769 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531955456_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:15,769 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531955456_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:15,770 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531955456_0001_00_000002 completed successfully
2016-05-18 00:39:15 Completed Dag: dag_1463531955456_0001_1
2016-05-18 00:39:15,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531955456_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:15,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531955456_0001_1_post
2016-05-18 00:39:15,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:15,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 1
2016-05-18 00:39:15,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:15,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true, with id=dag_1463531955456_0001_1
2016-05-18 00:39:15,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C] comprehensivesort/(1/1) ...sive/true_false_true_true, with id=dag_1463531955456_0001_1
2016-05-18 00:39:15,779 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1463531955456_0001
2016-05-18 00:39:15,781 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:15,781 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:15,781 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:15,781 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:15,781 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:15,782 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:15,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:15,784 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:15,784 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:15,785 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:15,785 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:15,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:15,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:15,785 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:15,788 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/EE54D3AC62FC42ED865D7D670847B2DA/C679D2F4A26A479F8A69BF57B9DCF47C/.tez/application_1463531955456_0001
2016-05-18 00:39:15,789 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true/_temporary
2016-05-18 00:39:15,789 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.352
2016-05-18 00:39:15,856 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:15,856 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:15,891 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-18 00:39:15,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-18 00:39:15,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:15,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:15,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:15,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:15,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true
2016-05-18 00:39:15,983 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:15,983 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:15,984 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"]
2016-05-18 00:39:15,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:15,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:15,985 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:15,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...nsive/true_true_true_true
2016-05-18 00:39:15,987 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:16,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:16,029 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:16,029 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:16,029 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:16,041 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001 doesn't exist and is created
2016-05-18 00:39:16,061 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:16,061 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:16,062 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd
2016-05-18 00:39:16,075 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531956029_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:16,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:16,075 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:16,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:16,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:16,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:16,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:16,078 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:16,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:16,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:16,080 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:16,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:16,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:16,081 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:16,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:16,081 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:16,082 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:16,082 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:16,082 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:16,082 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:16,084 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:16,085 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:16,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:16,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-log-dir
2016-05-18 00:39:16,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-log-dir/history.txt.appattempt_1463531956029_0001_000000, maxErrors=10
2016-05-18 00:39:16,086 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:16,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531956029_0001_000000, appSubmitTime=1463531956074, launchTime=1463531956074
2016-05-18 00:39:16,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:16,095 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40334
2016-05-18 00:39:16,095 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531956029
2016-05-18 00:39:16,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531956029_0001_000000, startTime=1463531956098
2016-05-18 00:39:16,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:16,161 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:16,161 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:16,162 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1463531956029_0001, dagName=[563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-05-18 00:39:16,179 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-05-18 00:39:16,179 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:16,180 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531956029_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-log-dir/dag_1463531956029_0001_1.dot
2016-05-18 00:39:16,180 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-log-dir/dag_1463531956029_0001_1-tez-dag.pb.txt
2016-05-18 00:39:16,186 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531956029_0001_1
2016-05-18 00:39:16,187 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,187 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-05-18 00:39:16 Running Dag: dag_1463531956029_0001_1
2016-05-18 00:39:16,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531956029_0001_1, submitTime=1463531956179
2016-05-18 00:39:16,197 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001/tez-conf.pb" } size: 36903 timestamp: 1463531956000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:16,197 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,197 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:16,198 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001/tez-conf.pb" } size: 36903 timestamp: 1463531956000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:16,198 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,198 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:16,198 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:16,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531956029_0001_1, initTime=1463531956198
2016-05-18 00:39:16,199 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956029_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:16,199 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:16,199 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1463531956029_0001, dagName=[563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-05-18 00:39:16,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:DAG_STARTED]: dagID=dag_1463531956029_0001_1, startTime=1463531956200
2016-05-18 00:39:16,200 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956029_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:16,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: BDA956541E65435FAE8D5C6DBDFECA1F : {090C0B09E6714138823113EB4755E5FC={InputName=090C0B09E6714138823113EB4755E5FC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:16,200 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: 090C0B09E6714138823113EB4755E5FC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:16,200 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,200 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,200 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,201 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,201 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:16,201 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:16,205 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,205 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531956029_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:16,206 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C], numAdditionalOutputs=1
2016-05-18 00:39:16,206 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=08E4C54B39CB4D919389B77C9AFBCE08, vertex=vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:16,208 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for A00E931D511A4C9CB0BE24DB500AF15C:08E4C54B39CB4D919389B77C9AFBCE08 using oldmapred API
2016-05-18 00:39:16,208 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=08E4C54B39CB4D919389B77C9AFBCE08, vertexName=A00E931D511A4C9CB0BE24DB500AF15C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:16,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, vertexId=vertex_1463531956029_0001_1_01, initRequestedTime=1463531956201, initedTime=1463531956206, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:16,209 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:16,210 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: 090C0B09E6714138823113EB4755E5FC on vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,212 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input 090C0B09E6714138823113EB4755E5FC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:16,213 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:16,213 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:16,213 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:16,214 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:16,214 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: 090C0B09E6714138823113EB4755E5FC on vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,215 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:16,215 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {090C0B09E6714138823113EB4755E5FC=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:16,219 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] parallelism set to 1
2016-05-18 00:39:16,219 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:16,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, vertexId=vertex_1463531956029_0001_1_00, initRequestedTime=1463531956200, initedTime=1463531956219, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:16,219 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] with distanceFromRoot: 0
2016-05-18 00:39:16,219 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:16,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531956029_0001_1_00, startRequestedTime=1463531956201, startedTime=1463531956219
2016-05-18 00:39:16,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531956029_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:16,220 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:16,220 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531956029_0001_1_00 for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] numStartedSources: 1 numSources: 1
2016-05-18 00:39:16,220 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] with distanceFromRoot: 1
2016-05-18 00:39:16,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531956029_0001_1_01, startRequestedTime=1463531956220, startedTime=1463531956220
2016-05-18 00:39:16,221 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:16,222 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: A00E931D511A4C9CB0BE24DB500AF15C with 0 source tasks and 1 pending tasks
2016-05-18 00:39:16,223 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: BDA956541E65435FAE8D5C6DBDFECA1F in vertex: A00E931D511A4C9CB0BE24DB500AF15C numBipartiteSourceTasks: 1
2016-05-18 00:39:16,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531956029_0001, with dag id: dag_1463531956029_0001_1
2016-05-18 00:39:16,224 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in BDA956541E65435FAE8D5C6DBDFECA1F
2016-05-18 00:39:16,224 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_STARTED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, taskId=task_1463531956029_0001_1_00_000000, scheduledTime=1463531956225, launchTime=1463531956225
2016-05-18 00:39:16,225 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,226 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:16,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531956029_0001_00_000001, launchTime=1463531956227
2016-05-18 00:39:16,229 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956029_0001_00_000001
2016-05-18 00:39:16,230 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956029_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,230 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531956029_0001_1_00_000000_0] started. Is using containerId: [container_1463531956029_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:16,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0, startTime=1463531956230, containerId=container_1463531956029_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:16 Starting to run new task attempt: attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,231 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,231 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,231 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:16,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: BDA956541E65435FAE8D5C6DBDFECA1F, VertexParallelism: 1, TaskAttemptID:attempt_1463531956029_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=090C0B09E6714138823113EB4755E5FC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A00E931D511A4C9CB0BE24DB500AF15C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:16,232 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:16,233 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,234 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531956029_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:16,234 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531956029_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:16,239 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:16,241 INFO input.MRInput (MRInput.java:initialize(440)) - 090C0B09E6714138823113EB4755E5FC using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:16,241 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: 090C0B09E6714138823113EB4755E5FC
2016-05-18 00:39:16,246 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDA956541E65435FAE8D5C6DBDFECA1F, ordinal: 0
2016-05-18 00:39:16,246 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDA956541E65435FAE8D5C6DBDFECA1F, mem on start (mb), free: 239, total: 332, max: 672
2016-05-18 00:39:16,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:16,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:16,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:16,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:16,248 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[090C0B09E6714138823113EB4755E5FC:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [A00E931D511A4C9CB0BE24DB500AF15C:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:16,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:16,249 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:16,249 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:16,249 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[090C0B09E6714138823113EB4755E5FC:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [A00E931D511A4C9CB0BE24DB500AF15C:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: 090C0B09E6714138823113EB4755E5FC being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: 090C0B09E6714138823113EB4755E5FC
2016-05-18 00:39:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:16,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:16,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:16,251 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,253 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:16,254 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - 090C0B09E6714138823113EB4755E5FC initialized RecordReader from event
2016-05-18 00:39:16,255 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:16,256 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 090C0B09E6714138823113EB4755E5FC
2016-05-18 00:39:16,256 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: DE17BC73F7FE4C14A00A2F16E34DC5EC
2016-05-18 00:39:16,256 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DE17BC73F7FE4C14A00A2F16E34DC5EC
2016-05-18 00:39:16,258 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - A00E931D511A4C9CB0BE24DB500AF15C using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@356aa8c0, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@6d6cf0a5, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:16,259 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for A00E931D511A4C9CB0BE24DB500AF15C: , UsingHashComparator=false
2016-05-18 00:39:16,272 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:16,273 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:16,273 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for A00E931D511A4C9CB0BE24DB500AF15C: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:16,273 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - A00E931D511A4C9CB0BE24DB500AF15C: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:16,273 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:16,274 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDA956541E65435FAE8D5C6DBDFECA1F, all 1 inputs ready in: 00:00:00.001
2016-05-18 00:39:16,278 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDA956541E65435FAE8D5C6DBDFECA1F, mem on close (mb), free: 137, total: 332, max: 672
2016-05-18 00:39:16,278 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - A00E931D511A4C9CB0BE24DB500AF15C: Starting flush of map output
2016-05-18 00:39:16,278 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - A00E931D511A4C9CB0BE24DB500AF15C: Span0.length = 37, perItem = 24
2016-05-18 00:39:16,280 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - A00E931D511A4C9CB0BE24DB500AF15C: done sorting span=0, length=37, time=1
2016-05-18 00:39:16,280 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - A00E931D511A4C9CB0BE24DB500AF15C: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:16,282 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - A00E931D511A4C9CB0BE24DB500AF15C: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-local-dir/output/attempt_1463531956029_0001_1_00_000000_0_10155_0/file.out
2016-05-18 00:39:16,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:16 Completed running task attempt: attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,285 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:16,285 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531956029_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:16,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531956029_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531956274, Process_Duration=4, Process_End_Time=1463531956278, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:16,286 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531956029_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,286 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, taskAttemptId=attempt_1463531956029_0001_1_00_000000_0, creationTime=1463531956225, allocationTime=1463531956227, startTime=1463531956230, finishTime=1463531956286, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:16,287 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531956029_0001_1_00_000000_0
2016-05-18 00:39:16,287 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:16,287 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,288 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956029_0001_00_000001
2016-05-18 00:39:16,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_FINISHED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, taskId=task_1463531956029_0001_1_00_000000, startTime=1463531956230, finishTime=1463531956287, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463531956029_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531956274, Process_Duration=4, Process_End_Time=1463531956278, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:16,288 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:16,288 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_FINISHED]: vertexName=BDA956541E65435FAE8D5C6DBDFECA1F, vertexId=vertex_1463531956029_0001_1_00, initRequestedTime=1463531956200, initedTime=1463531956219, startRequestedTime=1463531956201, startedTime=1463531956219, finishTime=1463531956288, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531956274, Process_Duration=4, Process_End_Time=1463531956278, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531956230, firstTasksToStart=[ task_1463531956029_0001_1_00_000000 ], lastTaskFinishTime=1463531956286, lastTasksToFinish=[ task_1463531956029_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531956029_0001_1_00_000000 ], longestDurationTasks=[ task_1463531956029_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:16,289 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:16,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531956029_0001_00_000001, stoppedTime=1463531956289, exitStatus=0
2016-05-18 00:39:16,289 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] attempt: attempt_1463531956029_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:16,289 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:16,289 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A00E931D511A4C9CB0BE24DB500AF15C
2016-05-18 00:39:16,290 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=BDA956541E65435FAE8D5C6DBDFECA1F, destinationVertexName=A00E931D511A4C9CB0BE24DB500AF15C. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:16,290 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,290 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:16,290 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:16,289 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_STARTED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, taskId=task_1463531956029_0001_1_01_000000, scheduledTime=1463531956291, launchTime=1463531956291
2016-05-18 00:39:16,291 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,291 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531956029_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:16,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531956029_0001_00_000002, launchTime=1463531956293
2016-05-18 00:39:16,294 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956029_0001_00_000002
2016-05-18 00:39:16,294 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956029_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531956029_0001_1_01_000000_0] started. Is using containerId: [container_1463531956029_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:16 Starting to run new task attempt: attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,295 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,295 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0, startTime=1463531956294, containerId=container_1463531956029_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:16,295 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:16,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: A00E931D511A4C9CB0BE24DB500AF15C, VertexParallelism: 1, TaskAttemptID:attempt_1463531956029_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDA956541E65435FAE8D5C6DBDFECA1F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=08E4C54B39CB4D919389B77C9AFBCE08, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:16,296 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:16,296 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,298 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531956029_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:16,298 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956029_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:16,298 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531956029_0001_00_000001, Exiting
2016-05-18 00:39:16,298 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531956029_0001_00_000001
2016-05-18 00:39:16,299 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:16,299 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531956029_0001_00_000001
2016-05-18 00:39:16,300 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531956029_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,301 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531956029_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:16,301 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531956029_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:16,300 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:16,302 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531956029_0001_00_000001 completed successfully
2016-05-18 00:39:16,306 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A00E931D511A4C9CB0BE24DB500AF15C, ordinal: 1
2016-05-18 00:39:16,306 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A00E931D511A4C9CB0BE24DB500AF15C, mem on start (mb), free: 132, total: 332, max: 672
2016-05-18 00:39:16,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:16,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:16,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:16,308 INFO output.MROutput (MROutput.java:initialize(421)) - 08E4C54B39CB4D919389B77C9AFBCE08: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:16,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:16,308 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[BDA956541E65435FAE8D5C6DBDFECA1F:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [08E4C54B39CB4D919389B77C9AFBCE08:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput]
2016-05-18 00:39:16,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:16,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:16,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:16,309 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[BDA956541E65435FAE8D5C6DBDFECA1F:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912], [08E4C54B39CB4D919389B77C9AFBCE08:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0]
2016-05-18 00:39:16,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:16,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: BDA956541E65435FAE8D5C6DBDFECA1F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:16,310 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - BDA956541E65435FAE8D5C6DBDFECA1F: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:16,310 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - BDA956541E65435FAE8D5C6DBDFECA1F: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:16,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:16,311 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: BDA956541E65435FAE8D5C6DBDFECA1F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:16,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:16,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: BDA956541E65435FAE8D5C6DBDFECA1F
2016-05-18 00:39:16,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:16,312 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,315 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BDA956541E65435FAE8D5C6DBDFECA1F: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:16,316 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:16,316 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531956029_0001_1_00_000000_0_10155} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531956316, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:16,317 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : BDA956541E65435FAE8D5C6DBDFECA1F
2016-05-18 00:39:16,317 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: DE17BC73F7FE4C14A00A2F16E34DC5EC
2016-05-18 00:39:16,317 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"], id: 08E4C54B39CB4D919389B77C9AFBCE08
2016-05-18 00:39:16,317 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"]
2016-05-18 00:39:16,317 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DE17BC73F7FE4C14A00A2F16E34DC5EC, for 1 inputs
2016-05-18 00:39:16,317 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:16,318 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: BDA956541E65435FAE8D5C6DBDFECA1F, wasInterrupted=false
2016-05-18 00:39:16,318 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:16,321 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:16,323 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001_wd/localmode-local-dir/output/attempt_1463531956029_0001_1_00_000000_0_10155/file.out, len=732, isLocal=true
2016-05-18 00:39:16,323 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:16,323 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:16,323 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:16,324 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:16,324 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : BDA956541E65435FAE8D5C6DBDFECA1F
2016-05-18 00:39:16,324 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A00E931D511A4C9CB0BE24DB500AF15C, all 1 inputs ready in: 00:00:00.007
2016-05-18 00:39:16,324 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - BDA956541E65435FAE8D5C6DBDFECA1F: Shuffle Runner thread complete
2016-05-18 00:39:16,325 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - BDA956541E65435FAE8D5C6DBDFECA1F: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:16,330 INFO output.MROutput (MROutput.java:close(526)) - 08E4C54B39CB4D919389B77C9AFBCE08 closed
2016-05-18 00:39:16,330 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531956029_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:16,331 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,331 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A00E931D511A4C9CB0BE24DB500AF15C, mem on close (mb), free: 129, total: 332, max: 672
2016-05-18 00:39:16,331 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: BDA956541E65435FAE8D5C6DBDFECA1F
2016-05-18 00:39:16,331 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - BDA956541E65435FAE8D5C6DBDFECA1F: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:16,331 INFO output.MROutput (MROutput.java:close(526)) - 08E4C54B39CB4D919389B77C9AFBCE08 closed
2016-05-18 00:39:16,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:16 Completed running task attempt: attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,332 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:16,332 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531956029_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:16,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531956029_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531956324, Process_Duration=7, Process_End_Time=1463531956331, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:16,333 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531956029_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, taskAttemptId=attempt_1463531956029_0001_1_01_000000_0, creationTime=1463531956291, allocationTime=1463531956292, startTime=1463531956294, finishTime=1463531956333, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:16,333 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,333 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:16,333 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,333 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531956029_0001_1_01_000000_0
2016-05-18 00:39:16,333 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956029_0001_00_000002
2016-05-18 00:39:16,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:TASK_FINISHED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, taskId=task_1463531956029_0001_1_01_000000, startTime=1463531956294, finishTime=1463531956333, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463531956029_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531956324, Process_Duration=7, Process_End_Time=1463531956331, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:16,335 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:16,335 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C]
2016-05-18 00:39:16,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:VERTEX_FINISHED]: vertexName=A00E931D511A4C9CB0BE24DB500AF15C, vertexId=vertex_1463531956029_0001_1_01, initRequestedTime=1463531956201, initedTime=1463531956206, startRequestedTime=1463531956220, startedTime=1463531956220, finishTime=1463531956335, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531956324, Process_Duration=7, Process_End_Time=1463531956331, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531956294, firstTasksToStart=[ task_1463531956029_0001_1_01_000000 ], lastTaskFinishTime=1463531956333, lastTasksToFinish=[ task_1463531956029_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531956029_0001_1_01_000000 ], longestDurationTasks=[ task_1463531956029_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:16,336 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:16,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531956029_0001_00_000002, stoppedTime=1463531956336, exitStatus=0
2016-05-18 00:39:16,336 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:16,336 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,336 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531956029_0001_1_00 [BDA956541E65435FAE8D5C6DBDFECA1F]
2016-05-18 00:39:16,336 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531956029_0001_1
2016-05-18 00:39:16,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531956029_0001_1
2016-05-18 00:39:16,337 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 08E4C54B39CB4D919389B77C9AFBCE08 for vertex: vertex_1463531956029_0001_1_01 [A00E931D511A4C9CB0BE24DB500AF15C], outputName: 08E4C54B39CB4D919389B77C9AFBCE08
2016-05-18 00:39:16,338 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956029_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:16,338 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531956029_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-18 00:39:16,343 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:08E4C54B39CB4D919389B77C9AFBCE08 of vertex/vertexGroup:A00E931D511A4C9CB0BE24DB500AF15C isVertexGroupOutput:false
2016-05-18 00:39:16,343 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,345 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531956029_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:16,345 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956029_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:16,345 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531956029_0001_00_000002, Exiting
2016-05-18 00:39:16,345 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531956029_0001_00_000002
2016-05-18 00:39:16,345 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531956029_0001_00_000002
2016-05-18 00:39:16,345 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531956029_0001_1, queueSize=0
2016-05-18 00:39:16,345 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531956029_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:16,345 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531956029_0001_00_000002 completed successfully
2016-05-18 00:39:16,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956029_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531956029_0001_1, startTime=1463531956199, finishTime=1463531956343, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=300, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063912598, Process_Duration=11, Process_End_Time=2927063912609, Read_Duration=0, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:16,346 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531956029_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:16,346 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956029_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:16 Completed Dag: dag_1463531956029_0001_1
2016-05-18 00:39:16,346 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531956029_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:16,346 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531956029_0001_1_post
2016-05-18 00:39:16,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,346 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:16,346 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:16,347 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true, with id=dag_1463531956029_0001_1
2016-05-18 00:39:16,347 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3] comprehensivesort/(1/1) ...nsive/true_true_true_true, with id=dag_1463531956029_0001_1
2016-05-18 00:39:16,353 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1463531956029_0001
2016-05-18 00:39:16,355 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:16,355 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:16,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:16,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:16,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:16,355 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:16,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:16,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:16,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:16,356 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:16,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:16,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:16,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:16,358 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:16,361 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/563E140B1C924D27B05BE8E8731943C3/CAA13E14D81D4D99B9B03B1868422EA3/.tez/application_1463531956029_0001
2016-05-18 00:39:16,361 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true/_temporary
2016-05-18 00:39:16,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.378
2016-05-18 00:39:16,428 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-18 00:39:16,428 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-18 00:39:16,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-18 00:39:16,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-18 00:39:16,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:16,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-18 00:39:16,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-18 00:39:16,484 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt
2016-05-18 00:39:16,490 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true
2016-05-18 00:39:16,550 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting
2016-05-18 00:39:16,550 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:16,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"]
2016-05-18 00:39:16,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-05-18 00:39:16,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] executing total steps: 1
2016-05-18 00:39:16,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] allocating management threads: 1
2016-05-18 00:39:16,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] starting step: (1/1) ...sive/false_true_true_true
2016-05-18 00:39:16,553 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-18 00:39:16,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] tez session mode enabled: true
2016-05-18 00:39:16,570 INFO client.TezClient (TezClient.java:<init>(174)) - Tez Client Version: [ component=tez-api, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:16,570 INFO client.TezClient (TezClient.java:start(394)) - Session mode. Starting session.
2016-05-18 00:39:16,570 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(164)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-18 00:39:16,579 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(122)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001 doesn't exist and is created
2016-05-18 00:39:16,593 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(332)) - DAGAppMaster thread has been created
2016-05-18 00:39:16,593 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-18 00:39:16,593 INFO client.LocalClient (LocalClient.java:run(282)) - Using working directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd
2016-05-18 00:39:16,602 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(363)) - Created DAGAppMaster for application appattempt_1463531956570_0001_000000, versionInfo=[ component=tez-dag, version=0.8.2, revision=2d46dcefa492a6957cfa972007039f7e188760ce, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20160112-1834 ]
2016-05-18 00:39:16,603 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(80)) - Trying to locate HadoopShimProvider for hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:16,603 INFO shim.HadoopShimsLoader (HadoopShimsLoader.java:<init>(139)) - Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=2.6.4, majorVersion=2, minorVersion=6
2016-05-18 00:39:16,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(436)) - AM Level configured TaskSchedulers: [0:TezUber:null]
2016-05-18 00:39:16,604 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(437)) - AM Level configured ContainerLaunchers: [0:TezUber:null]
2016-05-18 00:39:16,605 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(438)) - AM Level configured TaskCommunicators: [0:TezUber:null]
2016-05-18 00:39:16,605 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(447)) - Comparing client version with AM version, clientVersion=0.8.2, AMVersion=0.8.2
2016-05-18 00:39:16,606 INFO app.TaskCommunicatorManager (TaskCommunicatorManager.java:createUberTaskCommunicator(192)) - Creating Default Local Task Communicator
2016-05-18 00:39:16,607 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap
2016-05-18 00:39:16,607 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker
2016-05-18 00:39:16,607 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler
2016-05-18 00:39:16,607 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher
2016-05-18 00:39:16,608 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher
2016-05-18 00:39:16,608 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(529)) - Using concurrent dispatcher: false
2016-05-18 00:39:16,608 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher
2016-05-18 00:39:16,608 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher
2016-05-18 00:39:16,609 INFO common.AsyncDispatcher (AsyncDispatcher.java:registerAndCreateDispatcher(262)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:16,609 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler
2016-05-18 00:39:16,609 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager
2016-05-18 00:39:16,609 INFO launcher.TezContainerLauncherImpl (ContainerLauncherManager.java:createUberContainerLauncher(134)) - Creating LocalContainerLauncher
2016-05-18 00:39:16,611 INFO common.AsyncDispatcher (AsyncDispatcher.java:register(236)) - Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager
2016-05-18 00:39:16,612 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(78)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1, nodeUpdatesRescheduleEnabled: false
2016-05-18 00:39:16,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(57)) - Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService
2016-05-18 00:39:16,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-log-dir
2016-05-18 00:39:16,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-log-dir/history.txt.appattempt_1463531956570_0001_000000, maxErrors=10
2016-05-18 00:39:16,613 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(124)) - RecoveryService initialized with recoveryPath=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100
2016-05-18 00:39:16,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463531956570_0001_000000, appSubmitTime=1463531956602, launchTime=1463531956602
2016-05-18 00:39:16,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-18 00:39:16,618 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49732
2016-05-18 00:39:16,619 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:createUberTaskScheduler(557)) - Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1463531956570
2016-05-18 00:39:16,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463531956570_0001_000000, startTime=1463531956623
2016-05-18 00:39:16,624 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1945)) - In Session mode. Waiting for DAG over RPC
2016-05-18 00:39:16,693 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-18 00:39:16,693 INFO client.TezClient (TezClient.java:start(428)) - The url to track the Tez Session: N/A
2016-05-18 00:39:16,693 INFO client.TezClient (TezClient.java:submitDAGSession(470)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1463531956570_0001, dagName=[B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-05-18 00:39:16,709 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1351)) - Starting DAG submitted via RPC: [B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-05-18 00:39:16,709 INFO app.DAGAppMaster (DAGAppMaster.java:createDAG(985)) - Merging AM credentials into DAG credentials
2016-05-18 00:39:16,710 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(1076)) - Generating DAG graphviz file, dagId=dag_1463531956570_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-log-dir/dag_1463531956570_0001_1.dot
2016-05-18 00:39:16,710 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(1093)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-log-dir/dag_1463531956570_0001_1-tez-dag.pb.txt
2016-05-18 00:39:16,716 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531956570_0001_1
2016-05-18 00:39:16,716 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,716 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2399)) - Running DAG: [B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-05-18 00:39:16 Running Dag: dag_1463531956570_0001_1
2016-05-18 00:39:16,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463531956570_0001_1, submitTime=1463531956709
2016-05-18 00:39:16,725 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001/tez-conf.pb" } size: 36903 timestamp: 1463531956000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:16,726 INFO impl.VertexImpl (VertexImpl.java:setAdditionalInputs(3978)) - Setting 1 additional inputs for vertexvertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,726 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:16,727 INFO impl.VertexImpl (VertexImpl.java:<init>(928)) - Default container context for vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001/tez-conf.pb" } size: 36903 timestamp: 1463531956000 type: FILE visibility: APPLICATION],], environment: [[ CLASSPATH=$PWD/cascading-hadoop2-tez-3.1.0.jar/:$PWD/cascading-hadoop2-tez-3.1.0.jar/classes/:$PWD/cascading-hadoop2-tez-3.1.0.jar/lib/*::$PWD:$PWD/*:$HADOOP_CONF_DIR: ],[ LD_LIBRARY_PATH=$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA, vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4], Default Resources=<memory:1024, vCores:1>
2016-05-18 00:39:16,727 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4023)) - Setting 1 additional outputs for vertex vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,728 INFO impl.VertexImpl (VertexImpl.java:<init>(1016)) - Running vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] : TaskScheduler=0:TezUber, ContainerLauncher=0:TezUber, TaskCommunicator=0:TezUber
2016-05-18 00:39:16,728 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1593)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-18 00:39:16,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463531956570_0001_1, initTime=1463531956728
2016-05-18 00:39:16,728 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956570_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-18 00:39:16,730 INFO impl.DAGImpl (DAGImpl.java:transition(1768)) - Added additional resources : [[]] to classpath
2016-05-18 00:39:16,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:DAG_STARTED]: dagID=dag_1463531956570_0001_1, startTime=1463531956730
2016-05-18 00:39:16,730 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956570_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-18 00:39:16,730 INFO client.TezClient (TezClient.java:submitDAGSession(545)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1463531956570_0001, dagName=[B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-05-18 00:39:16,730 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2400)) - Root Inputs exist for Vertex: D7F95EF8EC8F49BB95C34104E46B70F2 : {B0549187F76B4E38B936DB3D9E491E9F={InputName=B0549187F76B4E38B936DB3D9E491E9F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-18 00:39:16,730 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2410)) - Starting root input initializer for input: B0549187F76B4E38B936DB3D9E491E9F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-18 00:39:16,730 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2550)) - Setting vertexManager to RootInputVertexManager for vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,731 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2721)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,731 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2726)) - Vertex will initialize from input initializer. vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,731 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3893)) - Starting 1 inputInitializers for vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,731 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-18 00:39:16,732 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(272)) - Starting InputInitializer for Input: B0549187F76B4E38B936DB3D9E491E9F on vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,732 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2562)) - Setting vertexManager to ShuffleVertexManager for vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,734 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(109)) - Input B0549187F76B4E38B936DB3D9E491E9F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7, groupingEnabled: true
2016-05-18 00:39:16,735 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(168)) - Grouping splits in Tez
2016-05-18 00:39:16,735 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(226)) - Desired splits: 1 too large. Desired splitLength: 221 Min splitLength: 52428800 New desired splits: 1 Total length: 221 Original splits: 2
2016-05-18 00:39:16,735 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(321)) - Desired numSplits: 1 lengthPerGroup: 221 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 221 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false
2016-05-18 00:39:16,735 INFO grouper.TezSplitGrouper (TezSplitGrouper.java:getGroupedSplits(539)) - Number of splits desired: 1 created: 1 splitsProcessed: 2
2016-05-18 00:39:16,735 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(1022)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-18 00:39:16,736 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(308)) - Succeeded InputInitializer for Input: B0549187F76B4E38B936DB3D9E491E9F on vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,736 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2760)) - Creating 1 tasks for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,736 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2779)) - Directly initializing vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531956570_0001_1_01, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false
2016-05-18 00:39:16,737 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2187)) - Setting up committers for vertex vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4], numAdditionalOutputs=1
2016-05-18 00:39:16,737 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(2202)) - Instantiating committer for output=24D19B31CB0743A4A2129593DBB113AC, vertex=vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-18 00:39:16,738 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(121)) - Committer for AD08833B725346E09DFF418BF28E0FD4:24D19B31CB0743A4A2129593DBB113AC using oldmapred API
2016-05-18 00:39:16,738 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(146)) - OutputCommitter for outputName=24D19B31CB0743A4A2129593DBB113AC, vertexName=AD08833B725346E09DFF418BF28E0FD4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-18 00:39:16,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, vertexId=vertex_1463531956570_0001_1_01, initRequestedTime=1463531956732, initedTime=1463531956737, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0, initGeneratedEventsCount=0
2016-05-18 00:39:16,739 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] transitioned from NEW to INITED due to event V_INIT
2016-05-18 00:39:16,740 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:16,740 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1644)) - Got updated RootInputsSpecs: {B0549187F76B4E38B936DB3D9E491E9F=forAllWorkUnits=true, update=[1]}
2016-05-18 00:39:16,740 INFO impl.VertexImpl (VertexImpl.java:setParallelismWrapper(1664)) - Vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] parallelism set to 1
2016-05-18 00:39:16,740 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(2972)) - Cannot init vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true
2016-05-18 00:39:16,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, vertexId=vertex_1463531956570_0001_1_00, initRequestedTime=1463531956730, initedTime=1463531956741, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1, initGeneratedEventsCount=1
2016-05-18 00:39:16,741 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] with distanceFromRoot: 0
2016-05-18 00:39:16,741 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION
2016-05-18 00:39:16,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531956570_0001_1_00, startRequestedTime=1463531956732, startedTime=1463531956741
2016-05-18 00:39:16,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1463531956570_0001_1_00, reconfigureDoneTime=0, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@b8338fe7, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true
2016-05-18 00:39:16,742 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:16,742 INFO impl.VertexImpl (VertexImpl.java:transition(2932)) - Source vertex started: vertex_1463531956570_0001_1_00 for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] numStartedSources: 1 numSources: 1
2016-05-18 00:39:16,742 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(2814)) - Triggering start event for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] with distanceFromRoot: 1
2016-05-18 00:39:16,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463531956570_0001_1_01, startRequestedTime=1463531956742, startedTime=1463531956742
2016-05-18 00:39:16,742 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] transitioned from INITED to RUNNING due to event V_START
2016-05-18 00:39:16,746 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(100)) - Starting 1 in D7F95EF8EC8F49BB95C34104E46B70F2
2016-05-18 00:39:16,746 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(519)) - OnVertexStarted vertex: AD08833B725346E09DFF418BF28E0FD4 with 0 source tasks and 1 pending tasks
2016-05-18 00:39:16,746 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,747 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(1051)) - Received configured notification : CONFIGURED for vertex: D7F95EF8EC8F49BB95C34104E46B70F2 in vertex: AD08833B725346E09DFF418BF28E0FD4 numBipartiteSourceTasks: 1
2016-05-18 00:39:16,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_STARTED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, taskId=task_1463531956570_0001_1_00_000000, scheduledTime=1463531956748, launchTime=1463531956748
2016-05-18 00:39:16,748 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,749 INFO node.PerSourceNodeTracker (PerSourceNodeTracker.java:nodeSeen(74)) - Adding new node 127.0.0.1:0 to nodeTracker 0
2016-05-18 00:39:16,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531956570_0001_00_000001, launchTime=1463531956750
2016-05-18 00:39:16,750 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956570_0001_00_000001
2016-05-18 00:39:16,751 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956570_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,751 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531956570_0001_1_00_000000_0] started. Is using containerId: [container_1463531956570_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-18 00:39:16,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0, startTime=1463531956751, containerId=container_1463531956570_0001_00_000001, nodeId=127.0.0.1:0
2016-05-18 00:39:16 Starting to run new task attempt: attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,753 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,753 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,753 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:16,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: D7F95EF8EC8F49BB95C34104E46B70F2, VertexParallelism: 1, TaskAttemptID:attempt_1463531956570_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B0549187F76B4E38B936DB3D9E491E9F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AD08833B725346E09DFF418BF28E0FD4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-18 00:39:16,753 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:16,754 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] submitted tez dag to app master: application_1463531956570_0001, with dag id: dag_1463531956570_0001_1
2016-05-18 00:39:16,757 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:16,758 INFO input.MRInput (MRInput.java:initialize(440)) - B0549187F76B4E38B936DB3D9E491E9F using newmapreduce API=false, split via event=true, numPhysicalInputs=1
2016-05-18 00:39:16,758 INFO input.MRInput (MRInput.java:initializeInternal(494)) - Initialized MRInput: B0549187F76B4E38B936DB3D9E491E9F
2016-05-18 00:39:16,758 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(496)) - TaskAttempt:attempt_1463531956570_0001_1_00_000000_0 sent events: (0-1).
2016-05-18 00:39:16,759 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531956570_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0
2016-05-18 00:39:16,761 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D7F95EF8EC8F49BB95C34104E46B70F2, ordinal: 0
2016-05-18 00:39:16,761 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7F95EF8EC8F49BB95C34104E46B70F2, mem on start (mb), free: 226, total: 333, max: 672
2016-05-18 00:39:16,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:16,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:16,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:16,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:16,762 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[B0549187F76B4E38B936DB3D9E491E9F:INPUT:0:org.apache.tez.mapreduce.input.MRInput], [AD08833B725346E09DFF418BF28E0FD4:OUTPUT:104857600:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput]
2016-05-18 00:39:16,762 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:16,762 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:16,762 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-18 00:39:16,762 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[B0549187F76B4E38B936DB3D9E491E9F:org.apache.tez.mapreduce.input.MRInput:INPUT:0:0], [AD08833B725346E09DFF418BF28E0FD4:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:104857600:104857600]
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: B0549187F76B4E38B936DB3D9E491E9F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: B0549187F76B4E38B936DB3D9E491E9F
2016-05-18 00:39:16,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:16,764 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,765 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(174)) - Processing split: TezGroupedSplit{wrappedSplits=[file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:0+110, file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt:110+111], wrappedInputFormatName='org.apache.hadoop.mapred.TextInputFormat', locations=[localhost], rack='null', length=221}
2016-05-18 00:39:16,765 INFO input.MRInput (MRInput.java:initFromEventInternal(653)) - B0549187F76B4E38B936DB3D9E491E9F initialized RecordReader from event
2016-05-18 00:39:16,767 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:16,768 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: B0549187F76B4E38B936DB3D9E491E9F
2016-05-18 00:39:16,768 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 23FB72C830764FDE86C0E1958CB64531
2016-05-18 00:39:16,768 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 23FB72C830764FDE86C0E1958CB64531
2016-05-18 00:39:16,771 INFO impl.ExternalSorter (ExternalSorter.java:<init>(199)) - AD08833B725346E09DFF418BF28E0FD4 using: memoryMb=100, keySerializerClass=class cascading.tuple.io.TuplePair, valueSerializerClass=cascading.tuple.hadoop.io.ValueTupleSerializer@3bc94186, comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@19ece314, partitioner=cascading.tuple.tez.util.GroupingSortingPartitioner, serialization=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-18 00:39:16,771 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(170)) - Setting up PipelinedSorter for AD08833B725346E09DFF418BF28E0FD4: , UsingHashComparator=false
2016-05-18 00:39:16,784 INFO impl.PipelinedSorter (PipelinedSorter.java:allocateSpace(245)) - Newly allocated block size=104857600, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=104857600, total=104857600
2016-05-18 00:39:16,784 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(189)) - Pre allocating rest of memory buffers upfront
2016-05-18 00:39:16,784 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(207)) - Setting up PipelinedSorter for AD08833B725346E09DFF418BF28E0FD4: , UsingHashComparator=false#blocks=1, maxMemUsage=104857600, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=104857600, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=100
2016-05-18 00:39:16,785 INFO impl.PipelinedSorter (PipelinedSorter.java:<init>(882)) - AD08833B725346E09DFF418BF28E0FD4: reserved.remaining()=104857600, reserved.metasize=16777216
2016-05-18 00:39:16,785 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-05-18 00:39:16,785 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D7F95EF8EC8F49BB95C34104E46B70F2, all 1 inputs ready in: 00:00:00.000
2016-05-18 00:39:16,790 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,790 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7F95EF8EC8F49BB95C34104E46B70F2, mem on close (mb), free: 124, total: 333, max: 672
2016-05-18 00:39:16,790 INFO impl.PipelinedSorter (PipelinedSorter.java:flush(629)) - AD08833B725346E09DFF418BF28E0FD4: Starting flush of map output
2016-05-18 00:39:16,791 INFO impl.PipelinedSorter (PipelinedSorter.java:end(995)) - AD08833B725346E09DFF418BF28E0FD4: Span0.length = 37, perItem = 24
2016-05-18 00:39:16,792 INFO impl.PipelinedSorter (PipelinedSorter.java:sort(904)) - AD08833B725346E09DFF418BF28E0FD4: done sorting span=0, length=37, time=1
2016-05-18 00:39:16,792 INFO impl.PipelinedSorter (PipelinedSorter.java:ready(1297)) - AD08833B725346E09DFF418BF28E0FD4: Heap = SpanIterator<0:36> (span=Span[16777216,888]),
2016-05-18 00:39:16,793 INFO impl.PipelinedSorter (PipelinedSorter.java:spill(546)) - AD08833B725346E09DFF418BF28E0FD4: Spilling to /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-local-dir/output/attempt_1463531956570_0001_1_00_000000_0_10161_0/file.out
2016-05-18 00:39:16,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:16 Completed running task attempt: attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,796 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0, askedToStop=false
2016-05-18 00:39:16,797 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531956570_0001_1_00_000000_0, stopRequested=false
2016-05-18 00:39:16,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531956570_0001_1_00_000000_0: Counters: 18 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][cascading.flow.SliceCounters Process_Begin_Time=1463531956785, Process_Duration=5, Process_End_Time=1463531956790, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-05-18 00:39:16,798 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531956570_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, taskAttemptId=attempt_1463531956570_0001_1_00_000000_0, creationTime=1463531956748, allocationTime=1463531956749, startTime=1463531956751, finishTime=1463531956798, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:16,798 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,798 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:16,799 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,799 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531956570_0001_1_00_000000_0
2016-05-18 00:39:16,799 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956570_0001_00_000001
2016-05-18 00:39:16,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_FINISHED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, taskId=task_1463531956570_0001_1_00_000000, startTime=1463531956751, finishTime=1463531956799, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463531956570_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531956785, Process_Duration=5, Process_End_Time=1463531956790, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-05-18 00:39:16,800 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:16,800 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_FINISHED]: vertexName=D7F95EF8EC8F49BB95C34104E46B70F2, vertexId=vertex_1463531956570_0001_1_00, initRequestedTime=1463531956730, initedTime=1463531956741, startRequestedTime=1463531956732, startedTime=1463531956741, finishTime=1463531956800, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, cascading.flow.SliceCounters, Process_Begin_Time=1463531956785, Process_Duration=5, Process_End_Time=1463531956790, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1463531956751, firstTasksToStart=[ task_1463531956570_0001_1_00_000000 ], lastTaskFinishTime=1463531956798, lastTasksToFinish=[ task_1463531956570_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531956570_0001_1_00_000000 ], longestDurationTasks=[ task_1463531956570_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:16,801 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:16,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531956570_0001_00_000001, stoppedTime=1463531956801, exitStatus=0
2016-05-18 00:39:16,801 INFO impl.VertexImpl (VertexImpl.java:transition(3279)) - Source task attempt completed for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] attempt: attempt_1463531956570_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-18 00:39:16,801 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:16,802 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,802 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(920)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AD08833B725346E09DFF418BF28E0FD4
2016-05-18 00:39:16,802 INFO impl.Edge (Edge.java:routingToBegin(232)) - Routing to begin for edge: EdgeInfo: sourceVertexName=D7F95EF8EC8F49BB95C34104E46B70F2, destinationVertexName=AD08833B725346E09DFF418BF28E0FD4. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true
2016-05-18 00:39:16,802 INFO impl.VertexImpl (VertexImpl.java:unsetTasksNotYetScheduled(1466)) - Routing pending task events for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,802 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(262)) - Ignoring stop request for containerId container_1463531956570_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-18 00:39:16,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_STARTED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, taskId=task_1463531956570_0001_1_01_000000, scheduledTime=1463531956803, launchTime=1463531956803
2016-05-18 00:39:16,803 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTaskEx(106)) - Scheduled vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_1463531956570_0001_00_000002, launchTime=1463531956804
2016-05-18 00:39:16,805 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956570_0001_00_000002
2016-05-18 00:39:16,805 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956570_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,805 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1322)) - TaskAttempt: [attempt_1463531956570_0001_1_01_000000_0] started. Is using containerId: [container_1463531956570_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-18 00:39:16,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0, startTime=1463531956805, containerId=container_1463531956570_0001_00_000002, nodeId=127.0.0.1:0
2016-05-18 00:39:16 Starting to run new task attempt: attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,806 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,806 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,806 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(306)) - Refreshing UGI since Credentials have changed. Credentials : #Tokens=1, #SecretKeys=0
2016-05-18 00:39:16,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(166)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: AD08833B725346E09DFF418BF28E0FD4, VertexParallelism: 1, TaskAttemptID:attempt_1463531956570_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7F95EF8EC8F49BB95C34104E46B70F2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=24D19B31CB0743A4A2129593DBB113AC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-18 00:39:16,807 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(93)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-18 00:39:16,807 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(65)) - Initializing task, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,810 INFO impl.VertexImpl (VertexImpl.java:getTaskAttemptTezEvents(3662)) - Sending attempt_1463531956570_0001_1_01_000000_0 1 events [0,1) total 1 vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,810 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531956570_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-18 00:39:16,810 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956570_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:16,810 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-18 00:39:16,810 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531956570_0001_00_000001, Exiting
2016-05-18 00:39:16,810 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531956570_0001_00_000001
2016-05-18 00:39:16,810 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531956570_0001_00_000001
2016-05-18 00:39:16,810 INFO task.TaskReporter (TaskReporter.java:heartbeat(292)) - Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1463531956570_0001_1_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1
2016-05-18 00:39:16,814 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531956570_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:16,814 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531956570_0001_00_000001 completed successfully
2016-05-18 00:39:16,815 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(339)) - Initial Memory required for SHUFFLE_BUFFER=211392912 based on INPUT_BUFFER_FACTOR=0.9, for final merged output=0, using factor: 0.0
2016-05-18 00:39:16,817 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AD08833B725346E09DFF418BF28E0FD4, ordinal: 1
2016-05-18 00:39:16,817 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD08833B725346E09DFF418BF28E0FD4, mem on start (mb), free: 119, total: 333, max: 672
2016-05-18 00:39:16,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(562)) - Initialized processor
2016-05-18 00:39:16,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 2 initializers to finish
2016-05-18 00:39:16,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(239)) - Waiting for 1 initializers to finish
2016-05-18 00:39:16,818 INFO output.MROutput (MROutput.java:initialize(421)) - 24D19B31CB0743A4A2129593DBB113AC: outputFormat=org.apache.hadoop.mapred.TextOutputFormat, using newmapreduce API=false
2016-05-18 00:39:16,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(252)) - All initializers finished
2016-05-18 00:39:16,819 INFO resources.MemoryDistributor (MemoryDistributor.java:logInitialRequests(275)) - InitialRequests=[24D19B31CB0743A4A2129593DBB113AC:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [D7F95EF8EC8F49BB95C34104E46B70F2:INPUT:211392912:org.apache.tez.runtime.library.input.OrderedGroupedKVInput]
2016-05-18 00:39:16,819 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:populateTypeScaleMap(239)) - ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1]
2016-05-18 00:39:16,819 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(266)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-18 00:39:16,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 211392912, TotalRequestedScaled: 1.9513191876923078E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.30
2016-05-18 00:39:16,820 INFO resources.MemoryDistributor (MemoryDistributor.java:logFinalAllocations(297)) - Allocations=[24D19B31CB0743A4A2129593DBB113AC:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [D7F95EF8EC8F49BB95C34104E46B70F2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:211392912:211392912]
2016-05-18 00:39:16,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(274)) - Starting Inputs/Outputs
2016-05-18 00:39:16,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(287)) - Input: D7F95EF8EC8F49BB95C34104E46B70F2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-18 00:39:16,821 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(147)) - D7F95EF8EC8F49BB95C34104E46B70F2: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true
2016-05-18 00:39:16,821 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(273)) - D7F95EF8EC8F49BB95C34104E46B70F2: MergerManager: memoryLimit=211392912, maxSingleShuffleLimit=52848228, mergeThreshold=190253616, ioSortFactor=100, postMergeMem=0, memToMemMergeOutputsThreshold=100
2016-05-18 00:39:16,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(308)) - Num IOs determined for AutoStart: 1
2016-05-18 00:39:16,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(310)) - Waiting for 1 IOs to start
2016-05-18 00:39:16,822 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(353)) - ShuffleScheduler running for sourceVertex: D7F95EF8EC8F49BB95C34104E46B70F2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true
2016-05-18 00:39:16,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:_callInternal(474)) - Started Input with src edge: D7F95EF8EC8F49BB95C34104E46B70F2
2016-05-18 00:39:16,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(323)) - AutoStartComplete
2016-05-18 00:39:16,829 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(70)) - Running task, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,831 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D7F95EF8EC8F49BB95C34104E46B70F2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0
2016-05-18 00:39:16,833 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-05-18 00:39:16,833 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 23FB72C830764FDE86C0E1958CB64531
2016-05-18 00:39:16,833 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"], id: 24D19B31CB0743A4A2129593DBB113AC
2016-05-18 00:39:16,833 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"]
2016-05-18 00:39:16,834 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 23FB72C830764FDE86C0E1958CB64531, for 1 inputs
2016-05-18 00:39:16,834 INFO orderedgrouped.ShuffleScheduler (ShuffleUtils.java:logIndividualFetchComplete(489)) - Completed fetch for attempt: {0, 0, attempt_1463531956570_0001_1_00_000000_0_10161} to DISK_DIRECT, csize=732, dsize=728, EndTime=1463531956834, TimeTaken=1, Rate=0.70 MB/s
2016-05-18 00:39:16,834 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(551)) - All inputs fetched for input vertex : D7F95EF8EC8F49BB95C34104E46B70F2
2016-05-18 00:39:16,835 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:16,835 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:callInternal(1289)) - Shutting down FetchScheduler for input: D7F95EF8EC8F49BB95C34104E46B70F2, wasInterrupted=false
2016-05-18 00:39:16,836 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(618)) - copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-18 00:39:16,836 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1005)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-18 00:39:16,837 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1103)) - Disk file=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001_wd/localmode-local-dir/output/attempt_1463531956570_0001_1_00_000000_0_10161/file.out, len=732, isLocal=true
2016-05-18 00:39:16,837 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1115)) - Merging 1 files, 732 bytes from disk
2016-05-18 00:39:16,837 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(1130)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-18 00:39:16,837 INFO impl.TezMerger (TezMerger.java:merge(644)) - Merging 1 sorted segments
2016-05-18 00:39:16,837 INFO impl.TezMerger (TezMerger.java:merge(749)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-05-18 00:39:16,838 INFO orderedgrouped.Shuffle (Shuffle.java:callInternal(332)) - merge complete for input vertex : D7F95EF8EC8F49BB95C34104E46B70F2
2016-05-18 00:39:16,838 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AD08833B725346E09DFF418BF28E0FD4, all 1 inputs ready in: 00:00:00.004
2016-05-18 00:39:16,838 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(417)) - D7F95EF8EC8F49BB95C34104E46B70F2: Shuffle Runner thread complete
2016-05-18 00:39:16,838 INFO input.OrderedGroupedKVInput (OrderedGroupedKVInput.java:createValuesIterator(292)) - D7F95EF8EC8F49BB95C34104E46B70F2: creating ValuesIterator with comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator, keyClass=cascading.tuple.io.TuplePair, valClass=cascading.tuple.io.ValueTuple
2016-05-18 00:39:16,844 INFO output.MROutput (MROutput.java:close(526)) - 24D19B31CB0743A4A2129593DBB113AC closed
2016-05-18 00:39:16,844 INFO impl.TaskImpl (TaskImpl.java:canCommit(671)) - attempt_1463531956570_0001_1_01_000000_0 given a go for committing the task output.
2016-05-18 00:39:16,844 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(79)) - Closing task, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,845 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AD08833B725346E09DFF418BF28E0FD4, mem on close (mb), free: 115, total: 333, max: 672
2016-05-18 00:39:16,845 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(278)) - Shutting down Shuffle for source: D7F95EF8EC8F49BB95C34104E46B70F2
2016-05-18 00:39:16,845 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:logProgress(76)) - D7F95EF8EC8F49BB95C34104E46B70F2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose
2016-05-18 00:39:16,845 INFO output.MROutput (MROutput.java:close(526)) - 24D19B31CB0743A4A2129593DBB113AC closed
2016-05-18 00:39:16,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:close(387)) - Joining on EventRouter
2016-05-18 00:39:16 Completed running task attempt: attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,845 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:run(86)) - Task completed, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0, askedToStop=false
2016-05-18 00:39:16,845 INFO task.TaskRunner2Callable (TaskRunner2Callable.java:callInternal(104)) - Cleaning up task attempt_1463531956570_0001_1_01_000000_0, stopRequested=false
2016-05-18 00:39:16,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(813)) - Final Counters for attempt_1463531956570_0001_1_01_000000_0: Counters: 36 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463531956838, Process_Duration=6, Process_End_Time=1463531956844, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-05-18 00:39:16,847 INFO task.TezTaskRunner2 (TezTaskRunner2.java:logAndReturnEndResult(473)) - TaskRunnerResult for attempt_1463531956570_0001_1_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,847 INFO task.TezChild (TezChild.java:run(265)) - TaskRunner2Result: TaskRunner2Result{endReason=SUCCESS, error=null, containerShutdownRequested=false}
2016-05-18 00:39:16,848 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend:
2016-05-18 00:39:16,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, taskAttemptId=attempt_1463531956570_0001_1_01_000000_0, creationTime=1463531956803, allocationTime=1463531956803, startTime=1463531956805, finishTime=1463531956848, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=
2016-05-18 00:39:16,848 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,848 INFO task.ContainerReporter (ContainerReporter.java:callInternal(57)) - Attempting to fetch new task for container container_1463531956570_0001_00_000002
2016-05-18 00:39:16,848 INFO impl.TaskImpl (TaskImpl.java:transition(1057)) - Task succeeded with attempt attempt_1463531956570_0001_1_01_000000_0
2016-05-18 00:39:16,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:TASK_FINISHED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, taskId=task_1463531956570_0001_1_01_000000, startTime=1463531956805, finishTime=1463531956848, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463531956570_0001_1_01_000000_0, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531956838, Process_Duration=6, Process_End_Time=1463531956844, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-05-18 00:39:16,849 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2024)) - Task Completion: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null
2016-05-18 00:39:16,849 INFO impl.VertexImpl (VertexImpl.java:checkTasksForCompletion(2037)) - All tasks have succeeded, vertex:vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4]
2016-05-18 00:39:16,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:VERTEX_FINISHED]: vertexName=AD08833B725346E09DFF418BF28E0FD4, vertexId=vertex_1463531956570_0001_1_01, initRequestedTime=1463531956732, initedTime=1463531956737, startRequestedTime=1463531956742, startedTime=1463531956742, finishTime=1463531956849, timeTaken=107, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463531956838, Process_Duration=6, Process_End_Time=1463531956844, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1463531956805, firstTasksToStart=[ task_1463531956570_0001_1_01_000000 ], lastTaskFinishTime=1463531956848, lastTasksToFinish=[ task_1463531956570_0001_1_01_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463531956570_0001_1_01_000000 ], longestDurationTasks=[ task_1463531956570_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-18 00:39:16,850 INFO impl.VertexImpl (VertexImpl.java:handle(1823)) - vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-18 00:39:16,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463531956570_0001_00_000002, stoppedTime=1463531956850, exitStatus=0
2016-05-18 00:39:16,850 INFO impl.DAGImpl (DAGImpl.java:transition(1927)) - Vertex vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-18 00:39:16,850 INFO impl.DAGImpl (DAGImpl.java:checkVerticesForCompletion(1261)) - Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,850 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1051)) - No output committers for vertex: vertex_1463531956570_0001_1_00 [D7F95EF8EC8F49BB95C34104E46B70F2]
2016-05-18 00:39:16,851 INFO impl.DAGImpl (DAGImpl.java:commitOrFinish(1098)) - Start writing dag commit event, dag_1463531956570_0001_1
2016-05-18 00:39:16,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463531956570_0001_1
2016-05-18 00:39:16,851 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956570_0001_1 transitioned from RUNNING to COMMITTING due to event DAG_VERTEX_COMPLETED
2016-05-18 00:39:16,851 INFO impl.DAGImpl (DAGImpl.java:call(1083)) - Committing output: 24D19B31CB0743A4A2129593DBB113AC for vertex: vertex_1463531956570_0001_1_01 [AD08833B725346E09DFF418BF28E0FD4], outputName: 24D19B31CB0743A4A2129593DBB113AC
2016-05-18 00:39:16,851 INFO app.TezTaskCommunicatorImpl (TezTaskCommunicatorImpl.java:getContainerTask(416)) - Container with id: container_1463531956570_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-18 00:39:16,852 INFO task.ContainerReporter (ContainerReporter.java:callInternal(67)) - Got TaskUpdate for containerId= container_1463531956570_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-18 00:39:16,852 INFO task.TezChild (TezChild.java:run(235)) - ContainerTask returned shouldDie=true for container container_1463531956570_0001_00_000002, Exiting
2016-05-18 00:39:16,852 INFO task.TezChild (TezChild.java:shutdown(381)) - Shutdown invoked for container container_1463531956570_0001_00_000002
2016-05-18 00:39:16,852 INFO task.TezChild (TezChild.java:shutdown(383)) - Shutting down container container_1463531956570_0001_00_000002
2016-05-18 00:39:16,852 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(285)) - ContainerExecutionResult for: container_1463531956570_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-18 00:39:16,852 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(289)) - Container: container_1463531956570_0001_00_000002 completed successfully
2016-05-18 00:39:16,854 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(260)) - Ignoring stop request for containerId: container_1463531956570_0001_00_000002
2016-05-18 00:39:16,858 INFO impl.DAGImpl (DAGImpl.java:commitCompleted(2192)) - Commit succeeded for output:outputName:24D19B31CB0743A4A2129593DBB113AC of vertex/vertexGroup:AD08833B725346E09DFF418BF28E0FD4 isVertexGroupOutput:false
2016-05-18 00:39:16,858 INFO impl.DAGImpl (DAGImpl.java:checkCommitsForCompletion(1313)) - Checking commits for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null
2016-05-18 00:39:16,860 INFO recovery.RecoveryService (RecoveryService.java:handle(327)) - DAG completed, dagId=dag_1463531956570_0001_1, queueSize=0
2016-05-18 00:39:16,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(115)) - [HISTORY][DAG:dag_1463531956570_0001_1][Event:DAG_FINISHED]: dagId=dag_1463531956570_0001_1, startTime=1463531956730, finishTime=1463531956858, timeTaken=128, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=240, AM_GC_TIME_MILLIS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=13, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927063913623, Process_Duration=11, Process_End_Time=2927063913634, Read_Duration=5, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-05-18 00:39:16,861 INFO impl.DAGImpl (DAGImpl.java:finished(1405)) - DAG: dag_1463531956570_0001_1 finished with state: SUCCEEDED
2016-05-18 00:39:16,861 INFO impl.DAGImpl (DAGImpl.java:handle(1170)) - dag_1463531956570_0001_1 transitioned from COMMITTING to SUCCEEDED due to event DAG_COMMIT_COMPLETED
2016-05-18 00:39:16 Completed Dag: dag_1463531956570_0001_1
2016-05-18 00:39:16,861 INFO app.DAGAppMaster (DAGAppMaster.java:handle(751)) - DAG completed, dagId=dag_1463531956570_0001_1, dagState=SUCCEEDED
2016-05-18 00:39:16,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(188)) - Redirecting log file based on addend: dag_1463531956570_0001_1_post
2016-05-18 00:39:16,861 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(203)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-18 00:39:16,861 INFO app.DAGAppMaster (DAGAppMaster.java:handle(800)) - Central Dispatcher queue size after DAG completion, before cleanup: 0
2016-05-18 00:39:16,861 INFO app.DAGAppMaster (DAGAppMaster.java:handle(802)) - Waiting for next DAG to be submitted.
2016-05-18 00:39:16,862 INFO app.DAGAppMaster (DAGAppMaster.java:handle(842)) - Cleaning up DAG: name=[B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true, with id=dag_1463531956570_0001_1
2016-05-18 00:39:16,862 INFO app.DAGAppMaster (DAGAppMaster.java:handle(852)) - Completed cleanup for DAG: name=[B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE] comprehensivesort/(1/1) ...sive/false_true_true_true, with id=dag_1463531956570_0001_1
2016-05-18 00:39:16,870 INFO client.TezClient (TezClient.java:stop(566)) - Shutting down Tez Session, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1463531956570_0001
2016-05-18 00:39:16,872 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(124)) - Received message to shutdown AM
2016-05-18 00:39:16,872 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:setShouldUnregisterFlag(895)) - TaskScheduler notified that it should unregister from RM
2016-05-18 00:39:16,872 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1301)) - No current running DAG, shutting down the AM
2016-05-18 00:39:16,872 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(907)) - DAGAppMasterShutdownHandler invoked
2016-05-18 00:39:16,872 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(916)) - Handling DAGAppMaster shutdown
2016-05-18 00:39:16,872 INFO app.DAGAppMaster (DAGAppMaster.java:run(936)) - Sleeping for 5 seconds before shutting down
2016-05-18 00:39:16,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(84)) - Stopping HistoryEventHandler
2016-05-18 00:39:16,873 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(196)) - Stopping RecoveryService
2016-05-18 00:39:16,873 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(199)) - Handle the remaining events in queue, queue size=0
2016-05-18 00:39:16,873 INFO recovery.RecoveryService (RecoveryService.java:run(173)) - EventQueue take interrupted. Returning
2016-05-18 00:39:16,874 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(221)) - Closing Summary Stream
2016-05-18 00:39:16,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-18 00:39:16,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-18 00:39:16,875 INFO rm.TaskSchedulerManager (TaskSchedulerManager.java:serviceStop(678)) - Shutting down AppCallbackExecutor
2016-05-18 00:39:16,876 INFO app.DAGAppMaster (DAGAppMaster.java:run(946)) - Calling stop for all the services
2016-05-18 00:39:16,877 INFO app.DAGAppMaster (DAGAppMaster.java:run(957)) - Exiting DAGAppMaster..GoodBye!
2016-05-18 00:39:16,878 INFO app.DAGAppMaster (DAGAppMaster.java:run(2084)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/tmp/.staging/B2C299FE175240458EC8EBDDBF08D5D4/CDB30D65046E4233822FC95DC2FE8BDE/.tez/application_1463531956570_0001
2016-05-18 00:39:16,879 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true/_temporary
2016-05-18 00:39:16,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [comprehensivesort] completed in: 00:00.329
2016-05-18 00:39:16,923 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false
2016-05-18 00:39:16,923 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true
2016-05-18 00:39:16,923 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true
2016-05-18 00:39:16,924 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false
2016-05-18 00:39:16,924 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true
2016-05-18 00:39:16,924 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true
2016-05-18 00:39:16,924 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true
2016-05-18 00:39:16,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true
2016-05-18 00:39:16,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false
2016-05-18 00:39:16,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true
2016-05-18 00:39:16,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false
2016-05-18 00:39:16,926 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true
2016-05-18 00:39:16,926 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true
2016-05-18 00:39:16,926 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true
2016-05-18 00:39:16,926 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false
2016-05-18 00:39:16,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true
2016-05-18 00:39:16,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false
2016-05-18 00:39:16,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false
2016-05-18 00:39:16,927 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false
2016-05-18 00:39:16,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false
2016-05-18 00:39:16,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false
2016-05-18 00:39:16,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false
2016-05-18 00:39:16,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true
2016-05-18 00:39:16,928 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.6.x-tez-0.8.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false